-- Logs begin at Sat 2022-09-24 20:01:15 UTC, end at Sat 2022-09-24 22:17:43 UTC. -- Sep 24 20:31:49.070183 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... Sep 24 20:31:49.080794 ncdv-master-1 devstack@keystone.service[99121]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 24 20:31:49 2022] *** Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: nodename: ncdv-master-1 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: machine: x86_64 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: clock source: unix Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: pcre jit disabled Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: detected number of CPU cores: 4 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: current working directory: / Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: detected binary path: /usr/bin/uwsgi-core Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: your processes number limit is 47680 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: your memory page size is 4096 bytes Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: detected max file descriptor number: 2048 Sep 24 20:31:49.082313 ncdv-master-1 devstack@keystone.service[99121]: lock engine: pthread robust mutexes Sep 24 20:31:49.083188 ncdv-master-1 devstack@keystone.service[99121]: thunder lock: enabled Sep 24 20:31:49.083188 ncdv-master-1 devstack@keystone.service[99121]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 24 20:31:49.083188 ncdv-master-1 devstack@keystone.service[99121]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 24 20:31:49.127497 ncdv-master-1 devstack@keystone.service[99121]: Python main interpreter initialized at 0x55b4c67b49c0 Sep 24 20:31:49.127497 ncdv-master-1 devstack@keystone.service[99121]: python threads support enabled Sep 24 20:31:49.127497 ncdv-master-1 devstack@keystone.service[99121]: your server socket listen backlog is limited to 100 connections Sep 24 20:31:49.127497 ncdv-master-1 devstack@keystone.service[99121]: your mercy for graceful operations on workers is 90 seconds Sep 24 20:31:49.127828 ncdv-master-1 devstack@keystone.service[99121]: mapped 403077 bytes (393 KB) for 2 cores Sep 24 20:31:49.127903 ncdv-master-1 devstack@keystone.service[99121]: *** Operational MODE: preforking *** Sep 24 20:31:49.128134 ncdv-master-1 devstack@keystone.service[99121]: *** uWSGI is running in multiple interpreter mode *** Sep 24 20:31:49.128134 ncdv-master-1 devstack@keystone.service[99121]: spawned uWSGI master process (pid: 99121) Sep 24 20:31:49.128328 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. Sep 24 20:31:49.133745 ncdv-master-1 devstack@keystone.service[99121]: spawned uWSGI worker 1 (pid: 99132, cores: 1) Sep 24 20:31:49.133745 ncdv-master-1 devstack@keystone.service[99121]: spawned uWSGI worker 2 (pid: 99133, cores: 1) Sep 24 20:31:49.133745 ncdv-master-1 devstack@keystone.service[99121]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 24 20:31:53.784064 ncdv-master-1 devstack@keystone.service[99133]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 20:31:53.784064 ncdv-master-1 devstack@keystone.service[99133]: from cryptography.utils import int_from_bytes Sep 24 20:31:53.784859 ncdv-master-1 devstack@keystone.service[99133]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 20:31:53.784859 ncdv-master-1 devstack@keystone.service[99133]: from cryptography.utils import int_from_bytes Sep 24 20:31:53.785696 ncdv-master-1 devstack@keystone.service[99132]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 20:31:53.785696 ncdv-master-1 devstack@keystone.service[99132]: from cryptography.utils import int_from_bytes Sep 24 20:31:53.786155 ncdv-master-1 devstack@keystone.service[99132]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 20:31:53.786155 ncdv-master-1 devstack@keystone.service[99132]: from cryptography.utils import int_from_bytes Sep 24 20:31:53.834089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 20:31:53.834264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 20:31:53.834452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] command line args: [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 20:31:53.834743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 20:31:53.834919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ================================================================================ {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 20:31:53.835169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] admin_token = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.835412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] config_dir = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.835539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 20:31:53.835625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.835738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 20:31:53.835825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] config_source = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.835923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] debug = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] command line args: [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 20:31:53.836117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 20:31:53.836391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ================================================================================ {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 20:31:53.836499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] admin_token = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] config_dir = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.836920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] config_source = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] debug = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_config_append = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.837954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_dir = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] insecure_debug = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_file = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.838930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_config_append = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_dir = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_file = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.839979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.840955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.841115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.841193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.841901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] publish_errors = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.842921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] notification_format = cadf {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.843912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] standard_threads = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] publish_errors = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] use_journal = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.844915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] use_json = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] use_stderr = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] use_syslog = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] standard_threads = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] strict_password_check = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.845881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.845979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] use_eventlog = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.846152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] use_journal = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.846363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] use_json = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] use_stderr = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.846678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] use_syslog = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.846904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] watch_log_file = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 20:31:53.846990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.847956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.848910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.849857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.850967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.851918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.852927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.853999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.854943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.855930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.856999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.857907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.858964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.859891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.860979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.861937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.862908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.863835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.864951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.enabled = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.865970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.866938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.867988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.868979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.869912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.external = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.870992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.password = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.871925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] auth.token = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.external = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.872957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.mapped = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.873869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.password = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] auth.token = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] catalog.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.874979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.875991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.876948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.877910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.878895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.879851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.880937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.881936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.882891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.883974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.884998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.885927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.886878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.887956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.888042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.888697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.888782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.888989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.889999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.890962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.891890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.password = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.892970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.893967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.894940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.895990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.896933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.897922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.898885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.899997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.900929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.901865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] policy.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.902949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] receipt.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.903844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.904449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.904605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.904760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.904837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] role.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] role.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.905838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] revoke.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.906982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] role.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] role.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.907987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] role.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] role.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.908945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.909981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.910968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.911905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.912893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.913955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.914998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.915920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.provider = fernet {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.916843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.917973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.connection = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] trust.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.918943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.919893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.connection = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.920853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.921936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 20:31:53.922421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.922884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.923117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.923357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.923589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.923834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.924066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.924307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.924534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.924770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.925003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 20:31:53.925170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=99132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 20:31:53.952452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.952672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.952888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.954351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.954602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.954828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.961393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.963307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.991477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.991632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.991703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:53.991777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.044978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.045779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.094113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.094333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.095800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.096033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.161808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.161958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.162174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.162391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.162634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.162852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.163271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.163501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.163722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.163940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.164155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.164369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.187893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.188663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.199579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.199877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.213955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.214745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=99132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 20:31:54.231526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.231736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 20:31:54.232029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.232133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.232279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 20:31:54.232362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 20:31:54.232554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.232813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.232917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.232993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 20:31:54.233067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.233272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.233354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.233534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.233789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.234044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.242726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.243002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.243269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.243472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.243555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.243748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.244015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.244296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.251325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.252116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.254167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.254451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.254766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.254984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.255074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.255927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.256967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.257156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.257427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.257696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.281819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.283204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.283292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.283567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.284653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.284934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.288190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.288470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.288748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.289431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.289711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.290098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.296058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.296446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.297300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.297597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.300660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.300952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.301236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.301526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.301608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.301806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.301968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.302817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.303092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.303378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.303649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.318115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.318393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.319397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.319677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.322240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.323526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.325449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.325777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.326946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.327364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.329957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.331562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.333097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.334680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.336312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.336609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.336890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.337170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.337449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.337747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.337840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.338047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.338328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.338746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.339036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.339320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.353886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.354174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.354453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.354742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.355830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.356119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.356392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.356655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.361637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.363182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.363704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.364706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.364982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.365329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.366886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.367162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.367602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.367883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.368180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.368474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.368857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.369793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.370059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.370348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.370666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.370954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.380501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.380781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.381063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.381344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.381726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.382019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.382313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.382991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.383275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.383555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.383831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.384105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.384388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.384669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.397561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.400248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.400452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.403520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.403660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.405612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.407092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.407216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.407380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.407777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.408648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.410090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.410368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.410697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.414096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.415530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.417957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.418370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.419586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.419846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.420060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.420429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.422797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.423087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.423370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.423651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.428193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.428481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.428761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.431535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.431860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.432166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.437220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.437515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.441909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=99132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 20:31:54.441993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.442970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.443128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.443385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=99132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 20:31:54.474429 ncdv-master-1 devstack@keystone.service[99133]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b4c67b49c0 pid: 99133 (default app) Sep 24 20:31:54.476865 ncdv-master-1 devstack@keystone.service[99132]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b4c67b49c0 pid: 99132 (default app) Sep 24 20:31:54.477783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200913-258d-410b-9ff5-8dd57aca97b3 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:31:54.478066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200913-258d-410b-9ff5-8dd57aca97b3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:31:54.478334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f200913-258d-410b-9ff5-8dd57aca97b3 None None] PATH_INFO: `/v3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:31:54.483109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Sat Sep 24 20:31:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:01.831258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:01.831763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:01.831847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:01.836008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:01.841509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:01.842036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:01.842036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:01.920685 ncdv-master-1 devstack@keystone.service[99133]: INFO dbcounter [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] Registered counter for database keystone Sep 24 20:32:01.939534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG oslo_db.sqlalchemy.engines [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99133) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 20:32:01.945613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writer thread running {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 20:32:02.124331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG passlib.handlers.bcrypt [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99133) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 20:32:02.124716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG passlib.handlers.bcrypt [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99133) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 20:32:02.182872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:02.184420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:02.317156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 24 20:32:02.664263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:02.740294 ncdv-master-1 devstack@keystone.service[99132]: INFO dbcounter [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] Registered counter for database keystone Sep 24 20:32:02.761288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG oslo_db.sqlalchemy.engines [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99132) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 20:32:02.768814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writer thread running {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 20:32:02.886523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acde9fe-3232-427e-a7f4-98eb2142fb62 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:02.891514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:02.892149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:02.892433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:02.892700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:02.902060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:02.954670 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 20:32:02.955016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151246-57aa-4586-8750-80bea50331b8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:02.965530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Sat Sep 24 20:32:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:32:02.972323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:02.990087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad5d45-49dd-413f-bdcf-8f8d79c03495 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:02.994610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:02.995736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:02.996015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:02.996285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] PATH_INFO: `/v3/projects/a8f45f34e55f4f50b20d34824a4e60af` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:02.998148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] RBAC: Authorizing `identity:get_project(project_id=a8f45f34e55f4f50b20d34824a4e60af)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:03.049972 ncdv-master-1 devstack@keystone.service[99133]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 20:32:03.050314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529a015-653c-4687-abd6-ce35fe3cedae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:03.056905 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Sat Sep 24 20:32:02 2022] GET /identity/v3/projects/a8f45f34e55f4f50b20d34824a4e60af => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:05.498745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f609-3195-4db3-8b27-f9c794150c35 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:05.499325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f609-3195-4db3-8b27-f9c794150c35 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:05.499325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0f609-3195-4db3-8b27-f9c794150c35 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:05.503384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:05.508753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:05.509138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:05.509308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:05.530100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:05.531641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:05.539413 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 20:32:05.861759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0f609-3195-4db3-8b27-f9c794150c35 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:05.873969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0f609-3195-4db3-8b27-f9c794150c35 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:05.878183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:05.878813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:05.879098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:05.879374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:05.886974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:05.889459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fe77d-7312-49fb-a179-fc73ca607a54 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:05.898121 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Sat Sep 24 20:32:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:32:05.903626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:05.912618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d5aa-142d-4eb2-90e9-db2c94fc9d83 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:05.916955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:05.917570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:05.917849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:05.918123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:05.920094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:05.921897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1617a-1cc5-4ae4-bc34-f37e18563b03 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:05.923448 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Sat Sep 24 20:32:05 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:08.388181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9247bc-a122-4038-93c2-13d38331272f None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:08.388181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9247bc-a122-4038-93c2-13d38331272f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:08.388787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9247bc-a122-4038-93c2-13d38331272f None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:08.389783 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:08.395364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:08.395803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:08.395896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:08.415920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:08.417110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:08.420572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524636e4-0b4b-4e22-99c8-9b9e4288054e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:08.420859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524636e4-0b4b-4e22-99c8-9b9e4288054e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:08.421140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524636e4-0b4b-4e22-99c8-9b9e4288054e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:08.422448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:08.427285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:08.427576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:08.427850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:08.472408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG passlib.handlers.bcrypt [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=99132) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 20:32:08.472796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG passlib.handlers.bcrypt [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=99132) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 20:32:08.523108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:08.524740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:08.607906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:08.626217 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:08.934627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:08.950363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:08.959282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.041806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8e700-1308-4128-ab92-faa0bf2416ed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.050983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2646e30-4d09-454e-a2c1-60d437db0ed0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.053269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.076733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.077371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.077650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.077921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.089024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:09.089507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.091128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.091495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.091787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:09.098389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:09.099443 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:09.100300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:09.105395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.162259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.163800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.186630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.204749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fa5aa-a19d-4674-b86e-f15390f8a1b3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.237846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.238449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.238759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.239063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.239730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:09.240041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.241432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.253367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:09.258982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.263324 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 20:32:09.268623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.271428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.360161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f3782-a4d2-4f00-a564-63a18a323519 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.367969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa4123-bc11-4489-a9fd-68f931e62238 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.394838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.395493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.396048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.396139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.397327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.398882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.400516 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:09.408080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.425412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.508531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.509294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.509576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.509849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.514815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592a364-4192-4f5b-b99a-267867114d46 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.530744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.532241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.548370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.548993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.549271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.549543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.550673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:32:09.550806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:09.551670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:32:09.552494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:32:09.552494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:32:09.552494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:09.552494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:09.552714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.554040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:09.557587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.559684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:09.616562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.644293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:09.662003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f069c-2249-451e-a8de-5f7a41642b10 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.664968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b31b2-9d50-403f-98aa-26e99e92c215 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:09.695487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.696100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.696380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.696651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.697410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:09.697716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.697814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:09.698350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:09.698666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:09.698940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:09.699134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89b120-2200-4576-b90b-b075aa4a96c6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.700171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.domain.id=default, user.id=b34a84022bc34fe0bb926f7c649f3ee4) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:09.700477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:09.701924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13065aaa-5be2-4e15-a571-9b81f8637cab None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:09.708593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/role_assignments?user.id=b34a84022bc34fe0bb926f7c649f3ee4&scope.domain.id=default&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:09.717208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:12.624075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfec9-30fd-40e9-8eab-0f2cd12ecc51 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:12.624614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfec9-30fd-40e9-8eab-0f2cd12ecc51 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:12.624614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfec9-30fd-40e9-8eab-0f2cd12ecc51 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:12.625923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:12.631941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4507bf-6463-4e20-9dcc-7321456c288a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:12.632870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4507bf-6463-4e20-9dcc-7321456c288a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:12.633336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4507bf-6463-4e20-9dcc-7321456c288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:12.664759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-be4507bf-6463-4e20-9dcc-7321456c288a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:12.666225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be4507bf-6463-4e20-9dcc-7321456c288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:12.761092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:12.990547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3631f-6811-454a-8f69-a4f90747b4bc None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:12.990883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3631f-6811-454a-8f69-a4f90747b4bc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:12.991057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3631f-6811-454a-8f69-a4f90747b4bc None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:12.992351 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:12.998582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772c5ae-c4bc-487a-b2a4-a0acc1d97d7e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:12.998831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772c5ae-c4bc-487a-b2a4-a0acc1d97d7e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:12.999151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772c5ae-c4bc-487a-b2a4-a0acc1d97d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.025145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45beb-0ea2-4bb1-b291-4711d2ea4113 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.025467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45beb-0ea2-4bb1-b291-4711d2ea4113 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.025688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b45beb-0ea2-4bb1-b291-4711d2ea4113 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.027483 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 14/25] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:13.028131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6772c5ae-c4bc-487a-b2a4-a0acc1d97d7e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:13.029606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6772c5ae-c4bc-487a-b2a4-a0acc1d97d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:13.035749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985ae3-8604-4c2d-a9cb-c82dc4e18ad8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.036030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985ae3-8604-4c2d-a9cb-c82dc4e18ad8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.036301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d985ae3-8604-4c2d-a9cb-c82dc4e18ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.067344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0d985ae3-8604-4c2d-a9cb-c82dc4e18ad8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:13.068557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d985ae3-8604-4c2d-a9cb-c82dc4e18ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:13.175503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:13.178895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 15/26] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:13.232569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 12/27] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:13.300823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f7f1-59be-42e8-9461-b376490da401 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.301097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f7f1-59be-42e8-9461-b376490da401 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.301305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7f7f1-59be-42e8-9461-b376490da401 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.302955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 16/28] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:13.312677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.312951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.313222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.339542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:13.340844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:13.405847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f7f1-59be-42e8-9461-b376490da401 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:13.441514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 13/29] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:13.523666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:13.556165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7f7f1-59be-42e8-9461-b376490da401 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:13.601951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:13.602628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.602913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.603189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.614770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:13.616249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:32:13.642604 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:13.643915 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:32:13.644710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:32:13.644710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:32:13.644710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:13.644710 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 17/30] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:13.651523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:13.661928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:13.810536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae381c7-4afa-4bbb-98f7-7776aa781ae9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:13.850843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:13.871815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:13.872442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.872727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.873004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.873700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:13.875208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:13.885154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4fb690-99ae-4295-9259-13286cc7ee40 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:13.911272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:13.957977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:13.958682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:13.958965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:13.959291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:13.959954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:13.960256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:13.961801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:13.975309 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 14/31] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:13.976083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.072262 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 18/32] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 20:32:13 2022] POST /identity/v3/roles => generated 219 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:14.073474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.096126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d4819-8415-4a60-9d6a-86e2218d59cb None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.191378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.192045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.192326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.192602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.233144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339f9ac-d6f1-42ce-a104-c2a7d4850a69 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.235633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:14.242134 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:14.243039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:14.243790 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:14.252171 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 15/33] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:14.252834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.287534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.287745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.288031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.288310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.288996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:14.292436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:14.316091 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 19/34] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 20:32:14 2022] POST /identity/v3/roles => generated 225 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:14.317230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.319945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:14.444212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:14.446436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7da18e-4ff3-4e5d-8f33-997fb873015b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.499284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.499916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.500191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.500461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.501366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:14.502922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60626ec6-9661-412b-830f-a9bd5ee74675 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:14.520374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e447-b60d-4d82-b78d-b1c31baad81c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.561902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.562552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.562847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.563117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.563796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:14.564100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:14.565488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d76ae6-6194-4fef-a3e1-703154b69f39 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:14.576423 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 20/35] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:14.577813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65c0e-7c66-4806-8fbf-0a8269addfc4 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.578099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65c0e-7c66-4806-8fbf-0a8269addfc4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.578375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c65c0e-7c66-4806-8fbf-0a8269addfc4 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.579666 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 21/36] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:14.580920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320c1d-ebe0-4d32-8fd3-2fba488993c2 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.581207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320c1d-ebe0-4d32-8fd3-2fba488993c2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.581626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320c1d-ebe0-4d32-8fd3-2fba488993c2 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.583125 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 22/37] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:14.585115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320c1d-ebe0-4d32-8fd3-2fba488993c2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.653175 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 16/38] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:32:14 2022] POST /identity/v3/projects => generated 301 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:14.654398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87104ba-9ee3-4a9c-af34-fc431526a7fd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.654726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87104ba-9ee3-4a9c-af34-fc431526a7fd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.654996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87104ba-9ee3-4a9c-af34-fc431526a7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.675775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e87104ba-9ee3-4a9c-af34-fc431526a7fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:14.677376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e87104ba-9ee3-4a9c-af34-fc431526a7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:14.708920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320c1d-ebe0-4d32-8fd3-2fba488993c2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.720009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:14.753530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.754189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.754582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.754783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.755940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:14.756789 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 17/39] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:14.757428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:14.759087 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 23/40] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:14 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:14.764495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.764495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.764495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.771787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:14.791134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:14.793608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:14.796530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:14.883560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 18/41] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:14.895790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbf284-4809-40e3-8f1d-1da92ce8a0c0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:14.936680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:14.937416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:14.937687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:14.937960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:14.939198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:14.941170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:14.942225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 24/42] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:14.949975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:15.050953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619b028-a610-4de1-9fb0-81a8e61a2d5d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:15.093844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:15.102539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:15.103245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:15.103509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:15.103788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] PATH_INFO: `/v3/domains/default/users/b34a84022bc34fe0bb926f7c649f3ee4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:15.106830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b34a84022bc34fe0bb926f7c649f3ee4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:15.108446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:15.120052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:15.125788 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 19/43] 10.222.0.52 () {58 vars in 1361 bytes} [Sat Sep 24 20:32:14 2022] PUT /identity/v3/domains/default/users/b34a84022bc34fe0bb926f7c649f3ee4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:15.153486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:15.217824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765b918-e1f5-4784-9742-c3e0e3893107 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:15.223275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:15.255960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:15.256605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:15.256890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:15.257167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:15.258313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:15.259844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:15.261461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 25/44] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:15.267312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:15.344574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf5548d-f99c-4247-b6b0-898af1c0209e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:15.375778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12414-ab72-459b-91f2-1d05c42e5a81 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:15.381489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:15.382239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:15.382689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:15.383059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:15.383905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:15.385436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f8f20-68bd-4a5b-bf65-1ca9420d9be2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:15.389779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:15.405603 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 20/45] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 20:32:15 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:32:15.419113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:15.419813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:15.420147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:15.420430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:15.421192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:15.422757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4a867-33fe-46c7-80c5-ca21200be294 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:15.454346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 26/46] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:32:15 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:32:17.545407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c968f1-2995-4771-af73-ad15770b7bb4 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:17.545921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c968f1-2995-4771-af73-ad15770b7bb4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:17.546002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c968f1-2995-4771-af73-ad15770b7bb4 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:17.547334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 21/47] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:17.552494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e8b8-d27c-43a3-9f05-ad85c0a4687f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:17.552889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e8b8-d27c-43a3-9f05-ad85c0a4687f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:17.553169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e8b8-d27c-43a3-9f05-ad85c0a4687f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:17.572320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d5b6e8b8-d27c-43a3-9f05-ad85c0a4687f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:17.573910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d5b6e8b8-d27c-43a3-9f05-ad85c0a4687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:17.644959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 27/48] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:17.903704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702ea8-a472-40ae-ad68-50c194e93f03 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:17.904025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702ea8-a472-40ae-ad68-50c194e93f03 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:17.904415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0702ea8-a472-40ae-ad68-50c194e93f03 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:17.905652 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 22/49] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:17.910919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:17.911205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:17.911480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:17.929579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:17.930788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:17.968044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702ea8-a472-40ae-ad68-50c194e93f03 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:17.997095 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 28/50] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:18.028716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:18.071339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0702ea8-a472-40ae-ad68-50c194e93f03 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.103677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.104316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.104587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.104860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.114600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:18.119401 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:18.120296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:18.121546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 23/51] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:18.126809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.151906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:18.228696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fc7d7-e196-4dfb-938d-b1dbfc74456e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.262682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.263287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.263562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.263829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.264486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:18.264782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.266164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.273849 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 29/52] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:18.279976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.331459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.382938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d5ad-37c0-4231-857a-b44a69fc768f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.401666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:18.420765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.421137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:18.421400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.421993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.422278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.423579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.425036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.426635 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 24/53] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:18 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:18.432112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.436945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb6b5-04c1-4ef2-9aae-9e102149601f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.470720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.471348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.471629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.471902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.473023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.474476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.476063 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 30/54] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:18.482186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.537541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d018f-0026-4188-8c6d-7d6e3829e9d4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.571308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.571945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.572229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.572502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.573623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.575622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.577184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 25/55] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:18.579537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe1fb-07d1-4872-9f27-de00ec208073 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.583393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:18.613112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.613747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.614031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.614305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.614974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.616416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab860d81-4c3a-454b-8dad-37ddaf42a60c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.641186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:18.644908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 31/56] 10.222.0.52 () {60 vars in 1140 bytes} [Sat Sep 24 20:32:18 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:18.690840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff244f01-86ad-4992-b5ee-908ae6f47606 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:18.726152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:18.726806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:18.727095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:18.727372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:18.728105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.domain.id=default, user.id=b34a84022bc34fe0bb926f7c649f3ee4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:18.728420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:18.729917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15c0ab-6fec-47f6-9af8-f8c00e654f47 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:18.739605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 26/57] 10.222.0.52 () {56 vars in 1322 bytes} [Sat Sep 24 20:32:18 2022] GET /identity/v3/role_assignments?user.id=b34a84022bc34fe0bb926f7c649f3ee4&scope.domain.id=default&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:21.054027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f215a-003c-4097-a4e3-21e1d593658b None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:21.054542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f215a-003c-4097-a4e3-21e1d593658b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:21.054542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f215a-003c-4097-a4e3-21e1d593658b None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:21.055855 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 32/58] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:21.061134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:21.061545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:21.061694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:21.080258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:21.081447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:21.145135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 27/59] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:21.474128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f215a-003c-4097-a4e3-21e1d593658b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:21.483349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:21.575248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f215a-003c-4097-a4e3-21e1d593658b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:21.608151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:21.608771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:21.609052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:21.609324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:21.610418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:21.611929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378011aa-976b-42ea-a1c5-35c9f08631ff None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:21.613457 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 33/60] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:21.618968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:21.667599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:21.717102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a2f2b-6bb5-485a-8dcf-82be49540565 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:21.749471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:21.750015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:21.750298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:21.750599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:21.751250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:21.752693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988687ca-7aaf-4024-bef2-124c88a8ca2b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:21.780420 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 28/61] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:21 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:25.462906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a08c8-03ce-4a25-86cc-d12af36b508c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.464054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a08c8-03ce-4a25-86cc-d12af36b508c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.464827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a08c8-03ce-4a25-86cc-d12af36b508c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.466375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 34/62] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:25.471846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cb27a-eb1b-405b-a932-9cac0c47652e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.473696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108aee-5724-408d-8eea-61ae9ab661e6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.474187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108aee-5724-408d-8eea-61ae9ab661e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.474688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108aee-5724-408d-8eea-61ae9ab661e6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.476162 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 35/63] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:25.476669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cb27a-eb1b-405b-a932-9cac0c47652e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.477157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cb27a-eb1b-405b-a932-9cac0c47652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.482226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.482795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.483572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.499597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-912cb27a-eb1b-405b-a932-9cac0c47652e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:25.500811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-912cb27a-eb1b-405b-a932-9cac0c47652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:25.505463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:25.506787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:25.580884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 29/64] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:25.582619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99035-244c-4e0a-994c-47083dbee346 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.583090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99035-244c-4e0a-994c-47083dbee346 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.583513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99035-244c-4e0a-994c-47083dbee346 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.585078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 30/65] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:25.591160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59f9f3-7fc5-4613-bce1-77b2b9f5010d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:25.591845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59f9f3-7fc5-4613-bce1-77b2b9f5010d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:25.592386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59f9f3-7fc5-4613-bce1-77b2b9f5010d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:25.607783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 36/66] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:25.611838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2d59f9f3-7fc5-4613-bce1-77b2b9f5010d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:25.613227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2d59f9f3-7fc5-4613-bce1-77b2b9f5010d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:25.642944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:25.693861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 31/67] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:26.002007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.005553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.038415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399000-fd27-4c1f-8ce2-63208190a0a4 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.038730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399000-fd27-4c1f-8ce2-63208190a0a4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.039009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6399000-fd27-4c1f-8ce2-63208190a0a4 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.040531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 32/68] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:26.046278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.046597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.046881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.077619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:26.078903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:26.141947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7436-3d37-4514-89a7-e71f90f6491e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.159728 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 33/69] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:26.160862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.181517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.182420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.183053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.183514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.194435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:26.199887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:26.200722 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:26.201882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 37/70] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:26.202531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.280712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.288541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.301489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450d81a-19aa-4fb8-a68a-602096efe111 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.322384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490681e-47a2-4dee-914f-4eb5fcedfc7b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.350181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.351100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.351805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.352084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.363014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.363649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.363928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.364197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.364800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:26.369659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:26.371402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 34/71] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:26.372629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.378637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3b633c04-dffb-400b-a8f3-a4bc83197b80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:26.383452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:26.384445 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:26.385356 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 38/72] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:26.386156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84b94-cea2-4fb0-9f3e-63eb7be89575 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.386156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84b94-cea2-4fb0-9f3e-63eb7be89575 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.387291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc84b94-cea2-4fb0-9f3e-63eb7be89575 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.388498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 39/73] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:26.392474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84b94-cea2-4fb0-9f3e-63eb7be89575 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.480223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ffbb-13af-443e-a927-1ee3ef7536d9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.503511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc84b94-cea2-4fb0-9f3e-63eb7be89575 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.515993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.516707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.516989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.517264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.517927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] RBAC: Adding query filter params (name=member) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:26.518231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:26.519682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:26.528038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.532783 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 35/74] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:26.533522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.543437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.544124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.544496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.544859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.545610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:26.546003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:26.547925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afa24f-7d41-4cd6-bd1d-2a964d263437 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:26.552407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.557621 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 40/75] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:26.559101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.559394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.559665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.577264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:26.578457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:26.638920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0a38f-b5a4-497e-821e-57e89bebf0dc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.645568 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 41/76] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:26.647418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.675404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.676116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.676402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.676897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.677540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:26.677801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:26.679331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:26.688167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 36/77] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:26.693207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.751677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3fe78-36d6-4e6d-a49e-49d23410fd4f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.771558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.782708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:26.792881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.793574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.793857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.794130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.804971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:26.805434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed171931-60a0-4951-80ca-53c5ec46f6fd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:26.809998 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:26.810959 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 42/78] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:26.815485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.845516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.846447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.846825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.847101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.863251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:26.864756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:26.870813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 37/79] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:26.870813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.919044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdf531-6759-45ca-93da-c3f9c3c48c50 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:26.952626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:26.953268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:26.953554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:26.953835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:26.955336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:26.958003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:26.959625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 43/80] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:26.960371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:26.985305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab70de-4cec-4a2f-9028-882ef66ddc3d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.007011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.018665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.019299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.019584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.019858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.020990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.022428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.024022 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 38/81] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.024022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.032157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.062915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d99bc5-8538-4fc8-ba0d-18db9df695da None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.099112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.099723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.100008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.100287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.100954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:27.101261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.102709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.110994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 44/82] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.115352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.137121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c79d-b145-4d8c-9b1b-189624012062 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.171522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.172134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.172415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.172684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.212975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e0ff7-f4f4-4725-a3f5-3ad7a1fd51f3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.221291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.245049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.246530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.249910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.250016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.250327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.250667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.251035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.316745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.318152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.342428 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 39/83] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:27.343110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.416754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 45/84] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:27.418156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.429132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.450072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3cd87-d058-4101-836f-3b5cc7bde1e8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.452842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.484830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.485459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.485733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.486006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.487122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.488562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.490278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 40/85] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.490885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.523926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0ea4-8195-4f27-aebe-7d124679eee0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.557663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.558258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.558561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.558845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.587916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad56548-f0e6-4c50-b012-64a1023294d6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.621122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.621748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.622028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.622300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.624429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.624741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.625886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.626250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.627495 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 41/86] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.628261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.639285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.673057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.730394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f40a3e-6c7f-4e14-967b-01e2129ce60e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.731254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 46/87] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:27.732161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.768003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.768631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.768911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.769181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.770381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.771896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.773561 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 42/88] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.775851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.831344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c86e7a-e038-4c88-be96-896af1fcb179 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.838439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.865958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.866613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.866898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.867174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.868298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.869765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.872052 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 47/89] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:27.876956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863652d-26b6-41ee-b366-2f8e51f211e9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:27.877622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.885450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:27.911938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:27.912549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:27.912826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:27.913100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:27.913743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:27.915310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:27.937408 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 43/90] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:27 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:27.938421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:27.987988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871ef0c-04a0-4bdc-a2c3-6a356fbcdb8d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.028481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.029112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.029391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.029676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.051374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f84a-0c99-4f51-a5a7-6ddb56a31fdd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.079018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:28.094885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.095554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.095840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.096223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.097295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.098803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.100493 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:28.103755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:28.113051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.114621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.140784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:28.230142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9fe9b-17c3-4357-aa60-72f7abe8e4fc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.230321 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 48/92] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:28.235075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:28.277613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.278247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.278559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.278842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.279570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:28.279882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.281472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.287590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 45/93] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:28 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:28.288437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:28.343822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:28.381098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de55881-9c4d-473a-8d99-ad0e5a0295b6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.419201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:28.428063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5f947-07d4-4284-805a-647510ce5e80 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.436747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.437721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.438223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.438749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.439599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=b34a84022bc34fe0bb926f7c649f3ee4) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:28.439980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.441552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.452208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 49/94] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:28 2022] GET /identity/v3/role_assignments?user.id=b34a84022bc34fe0bb926f7c649f3ee4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:28.452208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:28.485123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.486916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.487428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.487887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.489101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:28.489626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.492119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.500826 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:28 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=2413f08d373842418dcfc2305d420a05 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:28.661027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a09d3a-4e98-4ed8-bfab-70caec7fa7ab None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.710831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.711727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.712172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.712581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] PATH_INFO: `/v3/projects/37c01f723ac24e5d911c2a09e0da4245` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.714724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] RBAC: Authorizing `identity:get_project(project_id=37c01f723ac24e5d911c2a09e0da4245)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.716602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b8d55-5fdd-48aa-8da8-424d8155fe2b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.718548 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:28 2022] GET /identity/v3/projects/37c01f723ac24e5d911c2a09e0da4245 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:28.724644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:28.862796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9563970-9fee-401a-8dc1-de4d6b4c6164 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:28.910687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:28.911534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:28.911973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:28.912391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:28.913258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=37c01f723ac24e5d911c2a09e0da4245, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:28.913717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:28.915357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a54ec8-575f-4629-95fe-3404b6884802 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:28.922557 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:28 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=37c01f723ac24e5d911c2a09e0da4245&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:31.206749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002621e-a685-410b-ab1f-eeab5d7b5f66 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:31.207755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002621e-a685-410b-ab1f-eeab5d7b5f66 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:31.208264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002621e-a685-410b-ab1f-eeab5d7b5f66 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:31.209834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:31.232547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac644-0f32-465d-9db9-36e0f1dcfa9f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:31.233452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac644-0f32-465d-9db9-36e0f1dcfa9f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:31.233923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ac644-0f32-465d-9db9-36e0f1dcfa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:31.256942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8c6ac644-0f32-465d-9db9-36e0f1dcfa9f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:31.258436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8c6ac644-0f32-465d-9db9-36e0f1dcfa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:31.279728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:31.344988 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 48/99] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:31.540836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597450d-e992-424e-bdba-bcadbe301c8e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:31.541034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597450d-e992-424e-bdba-bcadbe301c8e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:31.543029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597450d-e992-424e-bdba-bcadbe301c8e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:31.544569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 52/100] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:31.550180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:31.550714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:31.551165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:31.572619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:31.574998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:31.664360 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 49/101] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:31.871222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597450d-e992-424e-bdba-bcadbe301c8e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:31.880882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.001019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597450d-e992-424e-bdba-bcadbe301c8e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.042338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.043283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.043565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.043839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.061288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-39e64236-6d58-4e4d-bcbe-e935ea22b5ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:32.066451 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:32.067400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:32.068141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 53/102] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:32.075371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.170307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.192466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae153642-50d7-4980-86f2-895e0f5db236 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.209421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afae4e-aa11-4f4f-a70b-42a6e095786c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.210359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afae4e-aa11-4f4f-a70b-42a6e095786c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.210908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afae4e-aa11-4f4f-a70b-42a6e095786c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.212512 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 54/103] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:32.227927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067243a6-345a-46b3-b729-ed62b02a4ea3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.228538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067243a6-345a-46b3-b729-ed62b02a4ea3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.229536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067243a6-345a-46b3-b729-ed62b02a4ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.235715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.236340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.236624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.236898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.237561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:32.237862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:32.239244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:32.254230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 50/104] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:32.262520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.271510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-067243a6-345a-46b3-b729-ed62b02a4ea3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:32.273042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-067243a6-345a-46b3-b729-ed62b02a4ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:32.402836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.406359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49eba15-f0b8-4e4b-8cbd-550dc256ec76 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.412020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 55/105] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:32.413600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8114a-8cdb-469f-9d51-7a554c08db64 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.414084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8114a-8cdb-469f-9d51-7a554c08db64 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.414583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8114a-8cdb-469f-9d51-7a554c08db64 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.416036 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 56/106] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:32.417139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8114a-8cdb-469f-9d51-7a554c08db64 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.487874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.488444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.488747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.489009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] PATH_INFO: `/v3/users/8b4bb1f464964e8fb5ec449d70853130` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.519573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4bb1f464964e8fb5ec449d70853130)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:32.521256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db445d-ed1f-4b03-b714-e91f780c64e0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:32.527094 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 51/107] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/users/8b4bb1f464964e8fb5ec449d70853130 => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:32.528791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.529293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.529763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.535719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8114a-8cdb-469f-9d51-7a554c08db64 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.543668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.560890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:32.562946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:32.575039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.575670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.575953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.576229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.587260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1c38a0bd-0bc9-4c6e-a8ff-e648c2c39143 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:32.592888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:32.593711 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:32.594549 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 57/108] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:32.599028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea491-9cd7-43c1-b9d0-458cd3c1ffd4 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.599028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea491-9cd7-43c1-b9d0-458cd3c1ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.599028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ea491-9cd7-43c1-b9d0-458cd3c1ffd4 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.599028 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 58/109] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:32.599028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea491-9cd7-43c1-b9d0-458cd3c1ffd4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.658764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 52/110] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:32.660464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.682960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.728018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ea491-9cd7-43c1-b9d0-458cd3c1ffd4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.768733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.769650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.770138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.770632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.835984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b2f42-2d86-4a12-9076-8fb2373b882d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:32.846743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:32.848596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d0b9-06de-460d-bd98-96c14213b02e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:32.873465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:32.874086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.874370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.874750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.875424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] RBAC: Adding query filter params (name=member) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:32.875710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:32.877094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad97de-7245-4175-8b27-2f3b9ac58b7e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:32.878406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.886629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 53/111] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:32.886629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.886629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.886629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.909428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:32.910646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:32.915192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:32.962249 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 59/112] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:32.968084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a42e0-9ae3-4776-b0a3-e4da412794d6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:32.968375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a42e0-9ae3-4776-b0a3-e4da412794d6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:32.968653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a42e0-9ae3-4776-b0a3-e4da412794d6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:32.970544 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 60/113] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:32.970652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a42e0-9ae3-4776-b0a3-e4da412794d6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:32.987357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 54/114] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:32.996272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.080879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a42e0-9ae3-4776-b0a3-e4da412794d6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.115815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c164bbf-0195-4eda-b6aa-b3ece90d107d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.116690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.117424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.117712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.118113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.129567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:33.134713 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:33.135870 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:33.136762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 61/115] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:33.137382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.144540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.155203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.155840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.156125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.156402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.157540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.157767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.159014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1f3a6-e1e7-4002-8e87-4407e42ea9f6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.160603 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 55/116] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:32 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.160603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.160603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.160603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.181022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:33.182770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:33.252149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244317-722b-459c-876f-23790b4f8bc6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.253410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 56/117] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:33.259369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.300923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.301533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.301886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.302077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.303238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.304692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.306313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 62/118] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.311443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.388771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e8697-b600-4948-8573-0e85f9732f10 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.415491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.416277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.429182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.429984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.430265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.430592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.435098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285a629-ed65-4d57-9053-ae7324505e25 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.441745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:33.446474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:33.447314 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:33.448670 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 57/119] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:33.449622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.478633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.480088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.480565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.481160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.482144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:33.482668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.484476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.493574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 63/120] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.498572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.562099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed246ae2-2a6f-4250-9cb7-c6cc55973a74 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.599879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.600889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.601181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.601459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.602655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.604009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efbd51-a475-47d4-89ff-5292256f6f80 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.604184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.605881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 58/121] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.606616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.640204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.640807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.641091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.641370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.642089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=8b4bb1f464964e8fb5ec449d70853130) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:33.642402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.643887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.649105 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 64/122] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/role_assignments?user.id=8b4bb1f464964e8fb5ec449d70853130&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.649874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.664140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.665151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.727152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb26977-ae96-4422-9b0d-40fcebacf2fb None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.764390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.765041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.765324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.765599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.766840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.768347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.770055 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 59/123] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.770795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2361d1-9d8d-4427-b3a8-d34f00564c51 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.770907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.812905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.813662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.813953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.814229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.814936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:33.815244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.816678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.826979 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 65/124] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.832559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.886550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d874d-9a30-4e2f-a929-2fb14b9ff644 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.909226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.912270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:33.923896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.924807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.925093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.925362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.926481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:33.927932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:33.929645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 60/125] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:33.930303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:33.950181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe5e16-297c-4595-b811-c4201bf00931 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:33.986806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:33.987435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:33.987713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:33.988076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:33.998808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:34.003942 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:34.004797 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:34.005541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 66/126] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:34.006165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.044588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0cf24-af4a-40fb-a80d-ee9aa466ffba None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.080401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.081177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.081461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.081760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.083755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, user_id=1fb4d47a50b04a298d4a8d7db55b1da4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.085188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.099902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 61/127] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:33 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:34.100741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.130800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf376038-3965-4e01-9dd1-0573f102a0f2 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.173168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.178835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.187186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e215e-f10d-45f8-8919-253878326310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.188170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e215e-f10d-45f8-8919-253878326310 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.188460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e215e-f10d-45f8-8919-253878326310 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.188740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e215e-f10d-45f8-8919-253878326310 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.189449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e215e-f10d-45f8-8919-253878326310 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.193714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e215e-f10d-45f8-8919-253878326310 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.224877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 67/128] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:32:34 2022] POST /identity/v3/projects => generated 302 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:34.225943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e215e-f10d-45f8-8919-253878326310 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.252240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210d16b-8196-4666-b202-a36eec37441c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.332378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.333837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.334341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.334876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.338791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.340741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.342927 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 62/129] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:34.343851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.351616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e215e-f10d-45f8-8919-253878326310 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.393011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.393933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.394440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.394896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] PATH_INFO: `/v3/projects/37c01f723ac24e5d911c2a09e0da4245` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.396255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] RBAC: Authorizing `identity:get_project(project_id=37c01f723ac24e5d911c2a09e0da4245)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.398119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.400122 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 68/130] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/projects/37c01f723ac24e5d911c2a09e0da4245 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:34.401596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.458928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fa90f-0ca7-42dc-ac32-66d0fc604220 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.468743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.476996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.498923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.499543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.499829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.500104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.500767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:34.501074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.502461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.517065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 63/131] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:34.518197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.544938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cff37-a79f-4581-b147-54a7af53934e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.583249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.584260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.584756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.585235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.586633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.588403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.590468 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:34.591485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.640911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407e1f5-1a14-457f-94d6-aaec747f4d98 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.679434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.680159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.680452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.680731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] PATH_INFO: `/v3/projects/37c01f723ac24e5d911c2a09e0da4245/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.683241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] RBAC: Authorizing `identity:create_grant(project_id=37c01f723ac24e5d911c2a09e0da4245, user_id=1fb4d47a50b04a298d4a8d7db55b1da4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.684947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.702515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 64/133] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:34 2022] PUT /identity/v3/projects/37c01f723ac24e5d911c2a09e0da4245/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:34.703526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.712223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285f838-8bae-43c1-8731-1aba337f2d0d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.749171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.758708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:34.782529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.783777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.784274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.784770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.786260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.787966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.789763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 70/134] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:34.807745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:34.884760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6d45b-ad7f-4deb-8035-a1c7a56642bf None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:34.955597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:34.956345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:34.956627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:34.956979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:34.965750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, user_id=1fb4d47a50b04a298d4a8d7db55b1da4, role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:34.966427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:34.991017 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 65/135] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:34 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/users/1fb4d47a50b04a298d4a8d7db55b1da4/roles/2413f08d373842418dcfc2305d420a05 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:35.006694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc499c-8a52-4e0d-8c2f-3626b09013dc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:35.048096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:35.049153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:35.049636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:35.050120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:35.051791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:35.053548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194e465-f9fa-4db2-bc66-aa88dbc5be24 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:35.055952 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 71/136] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:34 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:35.061998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:35.179119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:35.184397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ebdd5-5afa-4fac-abf9-67231ee2fcb9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:35.225822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:35.226419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:35.226740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:35.227021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/users/b34a84022bc34fe0bb926f7c649f3ee4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:35.229539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, user_id=b34a84022bc34fe0bb926f7c649f3ee4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:35.231039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db8b4e-6212-4385-ad4c-edcfaaffa25b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:35.258417 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 66/137] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:35 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/users/b34a84022bc34fe0bb926f7c649f3ee4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:37.248402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b0555-198e-4188-82b6-d9061e76c642 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:37.249431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b0555-198e-4188-82b6-d9061e76c642 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:37.249945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b0555-198e-4188-82b6-d9061e76c642 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:37.251786 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 72/138] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:37.256875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119d3f-f04b-4352-9ea0-7f7f13f2a0de None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:37.257699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119d3f-f04b-4352-9ea0-7f7f13f2a0de None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:37.258174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119d3f-f04b-4352-9ea0-7f7f13f2a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:37.281790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a2119d3f-f04b-4352-9ea0-7f7f13f2a0de None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:37.283377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2119d3f-f04b-4352-9ea0-7f7f13f2a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:37.365437 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 67/139] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:37.899108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e528c0f-e705-4337-9ba8-f74adbcf4fce None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:37.899794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e528c0f-e705-4337-9ba8-f74adbcf4fce None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:37.901059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e528c0f-e705-4337-9ba8-f74adbcf4fce None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:37.902702 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 73/140] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:37.908699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a2f-3bb8-4ca4-b3d4-468c2c7f1eba None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:37.909340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a2f-3bb8-4ca4-b3d4-468c2c7f1eba None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:37.909827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a2f-3bb8-4ca4-b3d4-468c2c7f1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:37.930332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d5198a2f-3bb8-4ca4-b3d4-468c2c7f1eba None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:37.932167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5198a2f-3bb8-4ca4-b3d4-468c2c7f1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:38.003998 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 68/141] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:38.027843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e528c0f-e705-4337-9ba8-f74adbcf4fce None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:38.091678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:38.130071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94502-2fa3-419b-a3e3-e599a6e41238 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.130389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94502-2fa3-419b-a3e3-e599a6e41238 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.130660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94502-2fa3-419b-a3e3-e599a6e41238 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.132135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 69/142] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:38.137641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3f86-0577-4512-8eb1-b51c6b7464ec None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.137926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3f86-0577-4512-8eb1-b51c6b7464ec None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.138203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3f86-0577-4512-8eb1-b51c6b7464ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.168626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-90ed3f86-0577-4512-8eb1-b51c6b7464ec None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:38.169731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-90ed3f86-0577-4512-8eb1-b51c6b7464ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:38.176752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e528c0f-e705-4337-9ba8-f74adbcf4fce None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:38.192942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:38.280146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:38.280848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.281068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.281353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.309895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 70/143] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:38.311121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9fe50-2da2-42ff-ae56-5c3d07c9cb96 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.311410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9fe50-2da2-42ff-ae56-5c3d07c9cb96 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.311771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9fe50-2da2-42ff-ae56-5c3d07c9cb96 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.312992 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 71/144] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:38.315539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:38.318348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442fe6-d15b-4af1-b8f1-56c65b618c89 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.318721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442fe6-d15b-4af1-b8f1-56c65b618c89 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.319241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442fe6-d15b-4af1-b8f1-56c65b618c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:38.320331 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:38.321194 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:38.321968 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:38.324726 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 74/145] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:38.330475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:38.357161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-75442fe6-d15b-4af1-b8f1-56c65b618c89 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:38.358598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75442fe6-d15b-4af1-b8f1-56c65b618c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:38.502745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684435-1f60-4b4d-a553-09b0a8950bfa None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:38.521145 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 72/146] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:38.522429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b752-2ec3-42ca-ad7e-e71bebc8fedb None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.522753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b752-2ec3-42ca-ad7e-e71bebc8fedb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.523029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919b752-2ec3-42ca-ad7e-e71bebc8fedb None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.524541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 73/147] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:38.529005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:38.535881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.536169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.536448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.543487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:38.544571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.545018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.545434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.546394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] RBAC: Adding query filter params (name=reader) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:38.547048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:38.548579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:38.560006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 75/148] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:38.568267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:38.578693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:38.579916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:38.718477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a1a08-bd05-40f7-8cac-dedb3fa3bdfa None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:38.747138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 74/149] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:38.749148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:38.768635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:38.779178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:38.779962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.780242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.780517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] PATH_INFO: `/v3/users/8b4bb1f464964e8fb5ec449d70853130` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.782265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4bb1f464964e8fb5ec449d70853130)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:38.784206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:38.786375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 76/150] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/users/8b4bb1f464964e8fb5ec449d70853130 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:38.787184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:38.890041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:38.938917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332310a1-8e25-42f9-90e5-58fdfd4e5060 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:38.968605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61274a18-fcd0-44de-8709-7cb85c6d4952 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:38.978052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:38.979261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:38.979647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:38.979929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:38.981270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:38.982745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:38.985427 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 75/151] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:38.986463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.014686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.015478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.015765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.016041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.030157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:39.036556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:39.037396 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:39.038141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 77/152] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:39.042520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.076404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.101873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c5a33-7e65-43a3-bacb-6a9b084a88a5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.135821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.136654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.136938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.137211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.138335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.139804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.141521 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 76/153] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:38 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:39.142265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.155454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7677b-77fc-4d19-b5a3-951d942b8e19 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.162223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.189340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.189991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.190263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.190559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.200560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:39.205682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:39.206551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:39.207300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 78/154] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:39.208121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.243020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20391b3f-9136-482c-b67a-0e1acc90331e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.278397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.279032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.279307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.279576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.280258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.281637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e2b4-d103-4ff5-bece-15c3dd21ba9c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.298272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.306681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 77/155] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:39 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:39.308733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4285-8c69-4aad-aded-2f28ec4a2297 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.309018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4285-8c69-4aad-aded-2f28ec4a2297 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.309303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd4285-8c69-4aad-aded-2f28ec4a2297 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.310829 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 78/156] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:39.311513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd4285-8c69-4aad-aded-2f28ec4a2297 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.316237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbb7a1-4d1f-4a05-aaa5-6cb8fa05a786 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.357444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.358056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.358330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.358681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.359493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:39.359795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.361422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.376555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 79/157] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:39.377497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.409434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.443890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd4285-8c69-4aad-aded-2f28ec4a2297 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.484898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.485661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.485948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.486287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.496153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c268a-ecf0-455f-8196-8d07f5036ddb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.498381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:39.503244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:39.504140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:39.504984 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 79/158] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:39.505657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.544651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.544831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.545060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.545259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/users/8b4bb1f464964e8fb5ec449d70853130/roles/4547e5401a2c4a38b03ce9c68699067d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.547750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, user_id=8b4bb1f464964e8fb5ec449d70853130, role_id=4547e5401a2c4a38b03ce9c68699067d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.549449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8585b2-6a27-4a8a-b54f-87ee37354442 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.569893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 80/159] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:39 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/users/8b4bb1f464964e8fb5ec449d70853130/roles/4547e5401a2c4a38b03ce9c68699067d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:39.570333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.570765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.571136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.576040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.594026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:39.595945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:39.643061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fc134-fea7-48d3-977c-847e03b026eb None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.694857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.695658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.696200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.696595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.697960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] RBAC: Adding query filter params (name=member) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:39.698373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.700564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.713652 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 80/160] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:39.714870 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 81/161] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:39.715639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.715991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.741899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.856459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6277cf-0780-4faa-aba0-e80de9ed5b41 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.873625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092774c-4acd-4a75-a955-dd2d430145ef None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:39.896693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:39.916796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.917446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.917734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.918007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.920135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.921595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.925939 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 81/162] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:39.926527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:39.945223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:39.945836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:39.946112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:39.946383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:39.947072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:39.947442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:39.948859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:39.965843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 82/163] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:39.966746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.108618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b0d1b-3a5d-4a8f-8f5a-8c1e1a937e55 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.118305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.153300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b848cf-e369-4edc-8ec9-4ba66ef7e7f1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.172880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.173523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.173798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.174069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.176408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:40.177857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:40.179495 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 82/164] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:40.180260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.225612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.226282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.226590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.226858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.274410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.314405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:40.316291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:40.330742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b43df-e68a-4e5b-b6ec-93c91d915652 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.376420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.377738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.378383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.378828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] PATH_INFO: `/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.380153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb4d47a50b04a298d4a8d7db55b1da4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:40.382405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:40.385409 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 83/165] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:40 2022] GET /identity/v3/users/1fb4d47a50b04a298d4a8d7db55b1da4 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:40.386384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.422981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.449843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 83/166] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:40.455703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.534868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b9df-a61c-4d89-9006-1be62cd6f1a7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.570625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.581972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.582221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9fea8-ebeb-4b41-9a4a-9a4b37738545 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.582849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.583219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.583582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.622023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.623004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.623415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.623845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.642996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:40.648164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:40.649200 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:40.650705 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:40.652738 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 84/167] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:40.653743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.691059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:40.693367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:40.704530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.816500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bcce6-2ff3-4f2d-aabd-82e01677f115 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:40.859841 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 84/168] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:40.861085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:40.872375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:40.873138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:40.873575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:40.873862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:40.891395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:40.976344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1260b8-4c97-4c24-b9b1-9c0bb163be91 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.007593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.009060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.014975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.015929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.016353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.016764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.018033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.019652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.021615 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 85/169] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:40 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.022569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.116610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.130425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.139666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18564e81-fc2c-438c-9edd-dc4b2b48eb10 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.177658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.178616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.179039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.179452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.180365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:41.180865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.183086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.192399 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 86/170] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.193458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.256985 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 85/171] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:41.257850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.382190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b006-cd9c-4fbe-aee1-292f504aded1 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.454676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.455448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.455727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.456026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] PATH_INFO: `/v3/projects/37c01f723ac24e5d911c2a09e0da4245` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.458322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081786c1-931a-46cc-82f8-227b64512685 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.460448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] RBAC: Authorizing `identity:get_project(project_id=37c01f723ac24e5d911c2a09e0da4245)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.461900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.465834 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 87/172] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/projects/37c01f723ac24e5d911c2a09e0da4245 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.475287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.521526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.523710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.524456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.524738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.525009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.526981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:41.528177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.530104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.537569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 86/173] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.538393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.543440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.693492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de641ac-f01d-42e1-b514-452af177885b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.721129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057f64d-e165-4d16-9257-5b3452370532 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:41.771234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.771884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.772163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.772435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] PATH_INFO: `/v3/users/b34a84022bc34fe0bb926f7c649f3ee4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.775828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] RBAC: Authorizing `identity:get_user(user_id=b34a84022bc34fe0bb926f7c649f3ee4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.777996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.780445 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 88/174] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/users/b34a84022bc34fe0bb926f7c649f3ee4 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.781236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.786787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:41.787431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:41.787707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:41.787980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:41.789144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:41.790704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:41.792862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 87/175] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:41.793632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:41.942861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.984193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:41.994983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf98f70-427e-45a1-995a-16954b610d40 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:42.011236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f269cd-f6d5-4b43-9c3f-8d08740dd4c7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:42.066458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:42.067088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:42.067362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:42.067629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:42.068343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=37c01f723ac24e5d911c2a09e0da4245, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:42.068642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:42.070079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:42.075354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:42.075963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:42.076236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:42.076504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:42.084458 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 89/176] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=37c01f723ac24e5d911c2a09e0da4245&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:42.090708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:42.191809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:42.193371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:42.208262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d1d1d-fecd-43b5-a003-e2cbd6356126 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:42.248268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:42.248795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:42.249067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:42.249330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:42.250047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=1fb4d47a50b04a298d4a8d7db55b1da4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:42.250346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:42.251825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:42.258012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 90/177] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:42 2022] GET /identity/v3/role_assignments?user.id=1fb4d47a50b04a298d4a8d7db55b1da4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=2413f08d373842418dcfc2305d420a05 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:42.331123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:42.372332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 88/178] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:42.377800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:42.419684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:42.531528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e756e14-be06-4872-af6c-185a3b032bbf None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:42.569143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:42.569791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:42.570066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:42.570335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:42.571559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:42.573971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869f8e-63e1-453c-9206-76d36d0bb018 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:42.576052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 91/179] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:42 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:42.581510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:42.756141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a558c-a8f2-4644-85eb-d936e9e8dd14 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:42.808109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:42.808660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:42.808934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:42.809211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:42.809957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=b34a84022bc34fe0bb926f7c649f3ee4) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:42.810266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:42.811780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2b270-5ae3-4be5-b273-67fb9f5cc09d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:42.818922 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 89/180] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:42 2022] GET /identity/v3/role_assignments?user.id=b34a84022bc34fe0bb926f7c649f3ee4&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:43.250096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe307a1-ad09-41a7-9864-32730b30c84e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.250806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe307a1-ad09-41a7-9864-32730b30c84e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.251298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe307a1-ad09-41a7-9864-32730b30c84e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.253097 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 92/181] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:43.258152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.258438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.258749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.280295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:43.281562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:43.320154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:43.339270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ce-ef6c-40f3-9a7f-8680a0952893 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.339632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ce-ef6c-40f3-9a7f-8680a0952893 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.339897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ce-ef6c-40f3-9a7f-8680a0952893 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.341357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 93/182] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:43.346581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07e232-e468-4672-9fa8-b7846742fc41 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.346867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07e232-e468-4672-9fa8-b7846742fc41 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.347142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07e232-e468-4672-9fa8-b7846742fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.366987 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 90/183] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:43.375277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bb07e232-e468-4672-9fa8-b7846742fc41 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:43.376515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bb07e232-e468-4672-9fa8-b7846742fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:43.466749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 94/184] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:32:43.719861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:43.815180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50d16-b684-4ce1-907d-777ab1176ed0 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.815453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50d16-b684-4ce1-907d-777ab1176ed0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.815760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50d16-b684-4ce1-907d-777ab1176ed0 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.817094 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 95/185] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:43.817841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f50d16-b684-4ce1-907d-777ab1176ed0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:43.826035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2b89c-9d8d-49f2-a330-838643fce3a0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:43.836488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:43.862158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:43.862829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.863106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.863374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.873598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-35ea5322-dd05-4c9b-bdd7-4731428dd8ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:43.878395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:43.879280 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:43.880036 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 91/186] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:43.880667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.881252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.881252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.900924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:43.902022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:43.923589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:43.935965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f50d16-b684-4ce1-907d-777ab1176ed0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:43.973272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:43.973894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:43.973998 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 92/187] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:43.974178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:43.974529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:43.983782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:43.986026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:43.991052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:43.991947 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:43.992723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 96/188] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:43.998246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.081188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.103826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f3932-715e-49be-8ffb-f2c28085d6f3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.105577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2549-c2aa-4279-82e4-a0164ee69ec7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.139526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.140133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.140413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.140688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.140953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.141345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:44.141548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.141646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.141935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.142208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.142898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] RBAC: Adding query filter params (name=reader) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:44.143050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.143191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.144551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c7558-fcd3-4655-ab74-f3b9243aa08f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.158339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 93/189] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:44.158339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e0ab-5d54-43eb-834d-f10eefea06f1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.158339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e0ab-5d54-43eb-834d-f10eefea06f1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.158339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313e0ab-5d54-43eb-834d-f10eefea06f1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.158339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 94/190] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:44.161886 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 97/191] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:44.161991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313e0ab-5d54-43eb-834d-f10eefea06f1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.162073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.181955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.272031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a0d45-7c05-47ee-840b-a81dd4f8de5f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.272566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313e0ab-5d54-43eb-834d-f10eefea06f1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.307227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.307939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.308213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.308482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.309594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.310221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.310538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.310804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] PATH_INFO: `/v3/users/8b4bb1f464964e8fb5ec449d70853130` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.311935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4bb1f464964e8fb5ec449d70853130)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.313376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48588a-4b58-4368-a29f-cb793b96a687 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.314987 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 98/192] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/users/8b4bb1f464964e8fb5ec449d70853130 => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:44.316093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.316372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.316645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.326820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.328279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.329953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 95/193] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:44.330746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.336102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:44.337273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:44.340117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.406654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 99/194] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:44.407498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.408073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.433649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a6520-90f4-4bd0-8b07-aabd21bac60d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.469994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.471021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.471448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.471880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.472998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.474450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.476219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 96/195] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:44.481419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.511107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb4423-5bc5-4384-b3fe-8e4256cf2299 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.548147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.548758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.549032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.549295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.559716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.586603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20bb9-607b-41ac-b4e8-5e5b8cb546da None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.622437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.623069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.623392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.623667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.624014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.625478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.645001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.703561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.705067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.736522 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 100/196] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:44.737344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.776343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.803110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 97/197] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:44.803110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.836466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991386f9-b6e9-4582-894e-30de46628b73 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.847656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:44.870447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.871081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.871364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.871626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.872268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:44.873704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:44.894139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 101/198] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:44 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:44.894998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:44.903618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750ef2-d461-402a-8e49-2ca7f77ae503 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:44.943209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:44.945127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:44.945405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:44.945721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:44.957840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:44.962891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:44.964072 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:44.964876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 98/199] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:44.965535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.016070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.025640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b898580-9b84-4016-bed8-d63750a50286 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.066775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.067783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.068136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.068479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.069645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.071166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.073088 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 102/200] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.075799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7aa57f-1c35-469f-afba-0a1c9fecdf3f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.075943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.077496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.119581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.121090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.121372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.121643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.122919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.124443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.127344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 99/201] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:44 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.128211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.198069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a089d-9dcd-4be6-a028-75db21e7b1dc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.239027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.239973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.240479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.240821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.241550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:45.241913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.243553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.249562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a8280-ec9e-4761-949d-37824fab32e9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.254021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 103/202] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.255070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.282698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.291855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.291855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.291855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.292162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.292746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, scope.project.id=b64a55cff23746df99bee214696e0d1e, user.id=8b4bb1f464964e8fb5ec449d70853130) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:45.293056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.295280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.301854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 100/203] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/role_assignments?user.id=8b4bb1f464964e8fb5ec449d70853130&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.302101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.354875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.385504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f16a-df5a-4813-b552-9257678e0add None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.427249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.427877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.428209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.428481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.429262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=119b624f945a488fb41990ec41d0959e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:45.429578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.430054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04726b7d-8f1f-4eac-96f5-2b05e0cd8851 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.431204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.438856 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 104/204] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/role_assignments?user.id=119b624f945a488fb41990ec41d0959e&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.471756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.472426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.472704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.472968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.474239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.475817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.477560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 101/205] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:45.484287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:45.613957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.633429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875898f-4c1a-4bee-a0ad-c0f91ec6c091 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:45.683191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:45.683814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:45.684092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:45.684364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:45.765509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:45.767455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:45.866104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:45.883060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 105/206] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:45.889143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:46.011937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1421f41-b211-4e28-8b9a-110a6b6b876c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:46.014390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:46.052851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:46.053489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:46.053803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:46.054017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:46.055168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:46.057180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689994e-640c-4489-b521-97fa1dc0cc2a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:46.061772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 102/207] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:45 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:46.065123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:46.171138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60254c3b-fe1a-4202-9882-cc6f3391a8ec None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:46.210952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:46.211561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:46.211837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:46.212169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:46.212917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=119b624f945a488fb41990ec41d0959e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:46.213223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:46.214672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a11f83-a4d7-4615-8cc5-fe9258a2fa00 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:46.220514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 106/208] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:46 2022] GET /identity/v3/role_assignments?user.id=119b624f945a488fb41990ec41d0959e&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=2413f08d373842418dcfc2305d420a05 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:47.663909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab397-7dfa-4023-8751-da09f53f3038 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.664449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab397-7dfa-4023-8751-da09f53f3038 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.664449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ab397-7dfa-4023-8751-da09f53f3038 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.666465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 103/209] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:47.672264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f9b73-6bf4-43ca-8780-f555a24ee383 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.672692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f9b73-6bf4-43ca-8780-f555a24ee383 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.672787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f9b73-6bf4-43ca-8780-f555a24ee383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.691840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ea5f9b73-6bf4-43ca-8780-f555a24ee383 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:47.693003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea5f9b73-6bf4-43ca-8780-f555a24ee383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:47.746710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:47.768819 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 107/210] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:47.810830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dac8a-75da-42e0-9f93-b8fe9b5b69a2 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.811152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dac8a-75da-42e0-9f93-b8fe9b5b69a2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.811374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71dac8a-75da-42e0-9f93-b8fe9b5b69a2 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.812993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 104/211] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:47.818159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f7c6-58a0-4e5e-ae15-0286d78c6a6b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.818443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f7c6-58a0-4e5e-ae15-0286d78c6a6b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.818751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f7c6-58a0-4e5e-ae15-0286d78c6a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.837435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d2e1f7c6-58a0-4e5e-ae15-0286d78c6a6b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:47.838631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d2e1f7c6-58a0-4e5e-ae15-0286d78c6a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:47.910706 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 108/212] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:47.961715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bdc99-8cdf-4ee0-be32-831a4a2ea5dc None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.962046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bdc99-8cdf-4ee0-be32-831a4a2ea5dc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.962550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bdc99-8cdf-4ee0-be32-831a4a2ea5dc None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.964134 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 105/213] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:47.969327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:47.969619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:47.969894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:47.989125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:47.990316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:48.059754 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 109/214] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:48.110768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bdc99-8cdf-4ee0-be32-831a4a2ea5dc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.215768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bdc99-8cdf-4ee0-be32-831a4a2ea5dc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.248781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.260601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.261748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.262033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.262308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.277099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:48.284664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:48.285522 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:48.286282 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 106/215] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:48.292054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.304973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.334128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.372812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be533-87ea-4dcb-9977-850b5d2857a7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.413540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.414207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.414479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.414806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.416096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:48.423765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:48.423765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 110/216] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:48.423765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.426273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c17223-ba98-4a61-9414-fa7b3e8a034f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.460740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.461335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.461609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.461874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.462628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:48.462930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:48.464315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:48.473217 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 107/217] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:48.478966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.532731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01094cc3-6b0d-425a-93cd-563f83c00578 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.544166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.569262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.569909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.570181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.570448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.581162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:48.584604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:48.586970 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:48.588750 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 111/218] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:48.594640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.600111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68beaf0-e4ab-4898-abbe-e34e473e58c1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.647720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.648338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.648614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.648883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.649530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:48.652055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:48.690522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 108/219] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:48 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:48.690522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.702351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0a344-8c61-42f1-a031-5fcc8c6bc465 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.742393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.743146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.743557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.743969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] PATH_INFO: `/v3/users/dc709f9e5c644ab498b746f584670923` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.761767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] RBAC: Authorizing `identity:get_user(user_id=dc709f9e5c644ab498b746f584670923)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:48.763346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0e5a4-2e95-439b-8242-314a983ad33f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:48.765224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 112/220] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/users/dc709f9e5c644ab498b746f584670923 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:48.766384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf738e-1ecb-4a47-a431-4978b74f8f6f None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.766689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf738e-1ecb-4a47-a431-4978b74f8f6f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.766954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf738e-1ecb-4a47-a431-4978b74f8f6f None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.768259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 113/221] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:48.771487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf738e-1ecb-4a47-a431-4978b74f8f6f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:48.797276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fe9f-3863-4046-a3bc-2a851acb2821 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.806641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.838601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.839341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.839624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.839898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.840581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:48.840887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:48.842321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97131184-5291-4c35-b5d9-f2fbad21fec4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:48.851330 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 109/222] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:48.857459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.857814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.858107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.877304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:48.878546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:48.887398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:48.894840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf738e-1ecb-4a47-a431-4978b74f8f6f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:48.938282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:48.939921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:48.939921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:48.939921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:48.966260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 110/223] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:48.967286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.031791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.034532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.073605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.087014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d33695-98a9-4c54-9937-ed6db2207ec7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.125883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.126927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.127381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.127821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.129272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.131120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.132962 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:49.142758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.157052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.157227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 114/225] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:49.165325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.258355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda59c6-0b33-47ea-b853-5d4bf1217ebb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.282202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98be54-ce5e-412a-93b6-5e2076e49f9c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.311147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.312095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.312988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.313505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.313978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.317813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.319242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.319959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.320344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.320936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.322188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.323789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.323966 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 112/226] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:49.327287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.327287 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:49.329244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.333086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.441087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.466739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714ff6e-96f9-43e0-8b46-185618733913 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.470249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6141b-5bf8-4de5-a011-c47e1823aed3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.508880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.509914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.510288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.510585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/119b624f945a488fb41990ec41d0959e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.512516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, user_id=119b624f945a488fb41990ec41d0959e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.513977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.514589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.515430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.515908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.516648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.518323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=dc709f9e5c644ab498b746f584670923) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:49.519006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.521167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.529953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 116/228] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/role_assignments?user.id=dc709f9e5c644ab498b746f584670923&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:49.531125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.534944 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 113/229] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:49 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/119b624f945a488fb41990ec41d0959e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:49.623266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.678739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dffdc5-8276-4e07-853f-885f890362a8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.726392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.727190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.727378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.727678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.745032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:49.751325 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:49.752278 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:49.753052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 117/230] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:49.763683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:49.923941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae9eb1-f8b1-4049-9427-7a79de6d6144 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:49.975832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:49.976808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:49.977307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:49.978472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:49.979574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:49.980074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:49.981663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:49.987490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:49.995100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 114/231] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:49.997309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:50.130565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebff63-d29b-4442-9720-73f0541cba3a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:50.157876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:50.183002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:50.183566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:50.183849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:50.184123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:50.185269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:50.186755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:50.188505 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 118/232] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:49 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:50.194009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:50.344511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73f16a-e734-4b0f-a681-32b197faf108 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:50.389320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:50.390361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:50.391029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:50.391524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:50.395155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:50.396826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38647f-a531-4fde-8470-f22fe117a518 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:50.399000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 115/233] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:50 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:50.407562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:50.550902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921dd77-f5c7-413e-839c-ac46acf5862f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:50.593892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:50.594597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:50.594882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:50.595153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/119b624f945a488fb41990ec41d0959e/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:50.597410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, user_id=119b624f945a488fb41990ec41d0959e, role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:50.599055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bab07b-77fb-4f38-9b54-a941a6fc9f9a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:50.625903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 119/234] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:50 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/119b624f945a488fb41990ec41d0959e/roles/2413f08d373842418dcfc2305d420a05 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:51.949314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f54f13-1f51-447e-8c38-4084874cc486 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:51.950244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f54f13-1f51-447e-8c38-4084874cc486 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:51.950741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f54f13-1f51-447e-8c38-4084874cc486 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:51.952266 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 116/235] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:51.969803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:51.970313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:51.970313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:51.993653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:51.994959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:52.044478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512923-9cc8-4ec9-983d-3c550a3be786 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.045130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512923-9cc8-4ec9-983d-3c550a3be786 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.045600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65512923-9cc8-4ec9-983d-3c550a3be786 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.047574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 117/236] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:52.050046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:52.052719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.053003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.053276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.081010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:52.083305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:52.084066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 120/237] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:52.185820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:52.192396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 118/238] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:52.559191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:52.578917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:52.686213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c8853-df10-4d0c-9998-2dbd6b3237e6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:52.686409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514274a-978f-4a55-8b01-0b850743ad70 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:52.723738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:52.724368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.724646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.724960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.725216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:52.725838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.726112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.726379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.728152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:52.729943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893d4db-573b-4556-b9d4-a83fb1ad4837 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:52.731530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 119/239] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:52.732695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b9dd2-cf4d-49bb-924e-01a53ed71e78 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.732973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b9dd2-cf4d-49bb-924e-01a53ed71e78 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.733240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b9dd2-cf4d-49bb-924e-01a53ed71e78 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.734561 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 120/240] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:52.735423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc8f-e636-4427-b7eb-b253d5b8ea9c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.735688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc8f-e636-4427-b7eb-b253d5b8ea9c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.735955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc8f-e636-4427-b7eb-b253d5b8ea9c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.736565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:52.737226 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 121/241] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:52.741484 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:52.742326 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:52.743176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:52.743772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc53ad-e3de-4809-81a6-faed87282776 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.743772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc53ad-e3de-4809-81a6-faed87282776 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.743772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc53ad-e3de-4809-81a6-faed87282776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.743945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 121/242] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:52.743945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:52.760868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-17fc53ad-e3de-4809-81a6-faed87282776 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:52.762041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-17fc53ad-e3de-4809-81a6-faed87282776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:52.785505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:52.824115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:52.839252 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 122/243] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:52.839252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.839252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.839252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.851431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:52.852819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:52.866678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743be53-192d-4128-a1a5-2437d71f815c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:52.912036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:52.912737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:52.913091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:52.913436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:52.914172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:52.915903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:52.931376 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 123/244] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:52.937433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:52.939648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 122/245] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:52 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:53.047463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505472-f835-4af2-bacf-2507158f5865 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.080506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.086173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.091373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.091688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.091927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.092631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:53.092937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.094376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.104779 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 124/246] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:53.110446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.190424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.247765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32995bb4-62af-48b6-995a-e22403c1e4b7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.249634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.290410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.291599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.291947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.292293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] PATH_INFO: `/v3/users/f310e99dafed4118a7064092835b5754` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.307103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] RBAC: Authorizing `identity:get_user(user_id=f310e99dafed4118a7064092835b5754)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.308822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.311120 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 123/247] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/users/f310e99dafed4118a7064092835b5754 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:53.316615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.329379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b9cdf-a48c-4b2a-9a33-38e84f760fd5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.368857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.369494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.369813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.370034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.382834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:53.388340 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:53.389296 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:53.390076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 125/248] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:53.391307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.443245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cb4c-259c-4cc9-a20c-80b4267bd697 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.451537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.488020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.489259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.489644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.490085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.517225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.530777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf09cba-7372-432e-9f09-21ba17ba3885 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.586518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.588322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.588954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.589533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.589848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.591354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.603122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:53.610742 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:53.611608 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:32:53.612371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:53.614893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 126/249] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:53.616165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.727857 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 124/250] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 20:32:53.729395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.750857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d86d7-6665-4100-b3bf-58a9c10259c4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.763834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.792731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.793443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.793797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.794142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.795827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:53.796207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.797818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.798308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:53.810444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 127/251] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:53.811414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.856974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0377d-c202-4d5b-af00-e480228d49a5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.900783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.901683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.902139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.902859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.904083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:53.904586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.906138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.917700 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 125/252] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:53.918795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:53.920994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865241e6-a71d-4497-9195-0ae30465d35d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:53.959144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:53.959882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:53.960234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:53.960599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:53.961987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:53.964155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:53.965682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 128/253] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:53.971269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.015756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:54.049823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de0b25-8e1e-4e92-a896-e52485980016 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.067300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:54.082514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5143e-15a4-4949-a5de-031656a6356a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.094560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.095456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.095917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.096381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] PATH_INFO: `/v3/users/dc709f9e5c644ab498b746f584670923` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.098384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] RBAC: Authorizing `identity:get_user(user_id=dc709f9e5c644ab498b746f584670923)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.100318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e0371-519b-4696-8033-0c7b1c31e2c9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.102288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 126/254] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/users/dc709f9e5c644ab498b746f584670923 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:54.103703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e22c7-45ae-470d-9210-4ff86640a94a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.104190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e22c7-45ae-470d-9210-4ff86640a94a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.104674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e22c7-45ae-470d-9210-4ff86640a94a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.106226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 127/255] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:54.107153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e22c7-45ae-470d-9210-4ff86640a94a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.121710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.122367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.122910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.122910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.124057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.125646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.127233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 129/256] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:53 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:54.127988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.232329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e22c7-45ae-470d-9210-4ff86640a94a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.267849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319d225-dc92-4f66-9ef8-9148caddcfc3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.269343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.270191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.270572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.270917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.271701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=f310e99dafed4118a7064092835b5754) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:54.272099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.273600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68095d62-9553-4638-bb3b-f72713090a33 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.276541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:54.287015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 128/257] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:54 2022] GET /identity/v3/role_assignments?user.id=f310e99dafed4118a7064092835b5754&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:54.289223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.289591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.290013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.307118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.307767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.308045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.308681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.310948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.312416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.314065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 130/258] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:54 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:54.314950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.335789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:54.337682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:54.439175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:54.460802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657fe97-cdcb-4524-9141-e9e066f3999a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.500691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.501231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.501510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.501813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.514231 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 129/259] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:54.515140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.591201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:54.610938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.612442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.660740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b43e6-8838-497a-a176-0e8edd17a652 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:54.749947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:54.750777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:54.751043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:54.751268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/dc709f9e5c644ab498b746f584670923/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:54.753210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, user_id=dc709f9e5c644ab498b746f584670923, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:54.754685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:54.790289 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 130/260] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:54 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/dc709f9e5c644ab498b746f584670923/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:54.843771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 131/261] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:54.849163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.895642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:54.924482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:55.037885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:55.075389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28ed3a-27d3-42de-835f-fae80c15ef1a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.089226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c160e79-23ac-428e-8657-ffe30ebbdf0c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.139574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.139779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.140227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.140335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.140510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.140603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.140768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.140865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.141906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:55.143398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:55.145371 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 131/262] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:54 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:55.152395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:55.161704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:55.167357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:55.168272 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:55.169062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 132/263] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:32:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:55.177259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:55.267202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a945841-c358-4a09-b0e0-62ed5492b267 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.270458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:55.307303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.307871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.308157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.308864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.309602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=119b624f945a488fb41990ec41d0959e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:55.309917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:55.311555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:55.317636 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 132/264] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:55 2022] GET /identity/v3/role_assignments?user.id=119b624f945a488fb41990ec41d0959e&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:55.393733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869b601-7142-4a4b-b86d-537a9f139ff0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.447335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.447967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.448244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.448520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.449202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:55.449512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:55.451087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:55.452746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:55.468740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 133/265] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:32:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:55.474315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:55.602972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926638d7-1f8f-40dd-87e9-2fef5e87e5b7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.652739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.653376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.653649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.653913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] PATH_INFO: `/v3/users/119b624f945a488fb41990ec41d0959e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.657383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] RBAC: Authorizing `identity:get_user(user_id=119b624f945a488fb41990ec41d0959e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:55.658856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:55.660443 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 133/266] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:55 2022] GET /identity/v3/users/119b624f945a488fb41990ec41d0959e => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:55.666186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:55.790343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a01f2e-c156-4a5e-9c04-ac88b5e07177 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:55.826687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:55.827289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:55.827561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:55.827830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:55.903600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:55.911396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:55.912869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:56.031592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 134/267] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:56.037277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:56.085544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:56.157744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f0d85-1138-472a-9445-7ee733255169 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:56.194015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:56.194789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.195191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.195497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.197699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:56.199188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168052e-695e-40e8-8383-7f43b377f8ab None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:56.200822 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 134/268] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:56 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:56.206933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:56.352062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:56.359329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e4286-6db0-4c1f-a277-afa2d1ff29aa None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:56.400222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:56.400835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.401118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.401394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.402130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=119b624f945a488fb41990ec41d0959e) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:56.402439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:56.403922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165d08e-7042-4362-9cc5-e422e11ceb32 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:56.412679 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 135/269] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:56 2022] GET /identity/v3/role_assignments?user.id=119b624f945a488fb41990ec41d0959e&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=2413f08d373842418dcfc2305d420a05 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:56.544641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cb0b-dc14-4d38-bd61-3a729117e0a2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.545171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cb0b-dc14-4d38-bd61-3a729117e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.545171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387cb0b-dc14-4d38-bd61-3a729117e0a2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.546634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 135/270] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:56.552575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2955ac-4afa-42b8-b8d5-0b7cc8bf7cfa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.552872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2955ac-4afa-42b8-b8d5-0b7cc8bf7cfa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.553134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2955ac-4afa-42b8-b8d5-0b7cc8bf7cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.582906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ea2955ac-4afa-42b8-b8d5-0b7cc8bf7cfa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:56.584152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea2955ac-4afa-42b8-b8d5-0b7cc8bf7cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:56.698584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 136/271] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:56.935831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cf73-945a-4e14-b7ff-fc5878c65f49 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.936197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cf73-945a-4e14-b7ff-fc5878c65f49 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.936399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25cf73-945a-4e14-b7ff-fc5878c65f49 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.937926 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 136/272] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:56.943478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:56.943765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:56.944042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:56.966288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:56.967577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:57.033640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25cf73-945a-4e14-b7ff-fc5878c65f49 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.050296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 137/273] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:57.136496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25cf73-945a-4e14-b7ff-fc5878c65f49 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.138904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:57.172659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.173283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.173555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.173819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.184488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:57.189346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:57.190185 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:57.190992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:32:57.195219 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 137/274] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:57.197132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.222611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:57.305702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623ccf7-b4f4-4550-b629-331c7e45c8c7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.340565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.341173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.341450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.341718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.342951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:57.343054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:57.344462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:57.353334 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 138/275] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:57.358834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.415941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.469985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a959c4-3eda-4e73-8c10-b0dd4403f156 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.502696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:57.512394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.513636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.513914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.514185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] PATH_INFO: `/v3/users/b1ccc792f7ff412785419498ee5eb867` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.527401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01693cd5-4595-43e5-92f4-f6e72b9ba818 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.531988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] RBAC: Authorizing `identity:get_user(user_id=b1ccc792f7ff412785419498ee5eb867)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:57.533473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:57.538569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 138/276] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:57 2022] GET /identity/v3/users/b1ccc792f7ff412785419498ee5eb867 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:57.542355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.569660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:57.575169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.576682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.577473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.578240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.579894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:57.581620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:57.583818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 139/277] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:57.595833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:57.660806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33c4b2-cab6-4ec0-aa03-7ba600110fbf None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.700687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.701442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.701861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.702249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.703191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, user.id=b1ccc792f7ff412785419498ee5eb867, scope.system=all) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:57.703599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:57.705091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f6cd5-2843-4066-af1e-eed11f60832c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:57.710374 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 139/278] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 20:32:57 2022] GET /identity/v3/role_assignments?user.id=b1ccc792f7ff412785419498ee5eb867&scope.system=all&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:57.717599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa7b64-b594-495f-ac44-6b1d5bc0e72a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.717846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa7b64-b594-495f-ac44-6b1d5bc0e72a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.718113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa7b64-b594-495f-ac44-6b1d5bc0e72a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.722518 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 140/279] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:32:57.725587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c746a5-f41c-4573-bf38-0eab61d5ce68 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.725865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c746a5-f41c-4573-bf38-0eab61d5ce68 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.726131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c746a5-f41c-4573-bf38-0eab61d5ce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.741705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd83eea-c4de-48af-a458-151229547b5f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:57.746815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60c746a5-f41c-4573-bf38-0eab61d5ce68 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:57.748033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60c746a5-f41c-4573-bf38-0eab61d5ce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:57.780255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:57.780789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.781069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.781342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.781999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:57.783806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed500710-d1ba-4abe-be45-297008519664 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:57.811499 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 140/280] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:32:57 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:32:57.832334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 141/281] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:57.944168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1002f-b9b2-4d6d-988c-27d3d48c5d5f None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.944555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1002f-b9b2-4d6d-988c-27d3d48c5d5f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.944791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1002f-b9b2-4d6d-988c-27d3d48c5d5f None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.947375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 141/282] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:32:57.953465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:57.953768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:57.954012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:57.973932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:32:57.975289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:32:57.988480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.058670 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 142/283] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:32:58.171762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1002f-b9b2-4d6d-988c-27d3d48c5d5f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.182638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.287831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1002f-b9b2-4d6d-988c-27d3d48c5d5f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:58.325174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:58.325731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:58.326009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:58.326280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:58.337706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:58.342730 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:58.343595 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:58.344408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 142/284] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:32:58.350023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.428987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.481383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20029fc-e7a0-4396-8f5a-96ad56f78054 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:58.507028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.549528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14e80a-e31d-4dfc-8535-e89443684835 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:58.568121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:58.569039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:58.569528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:58.570095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:58.572231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:58.572722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:58.574277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:58.590195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:58.591063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:58.591429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:58.591767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:58.593513 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 143/285] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:58.599106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.607123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:32:58.613560 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:32:58.614439 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:32:58.615241 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:32:58.624595 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 143/286] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:32:58.632672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.636982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.783449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a8452-e48d-45cd-b507-f36dde24f057 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:58.807379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d11e46-2a3e-44a3-997d-1dd08622c682 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:58.810703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.828734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:58.829368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:58.829661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:58.829929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:58.830649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:58.830952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:58.832350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:58.862726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:58.863411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:58.863749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:58.864074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] PATH_INFO: `/v3/users/f310e99dafed4118a7064092835b5754` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:58.865295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] RBAC: Authorizing `identity:get_user(user_id=f310e99dafed4118a7064092835b5754)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:58.867147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:58.869630 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 144/288] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:58.874476 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 144/288] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/users/f310e99dafed4118a7064092835b5754 => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:58.875864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.875970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:58.969343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:58.988455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7cb0b-4460-4cab-b68a-4098b4e04428 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.027963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.028931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.029381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.029826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] PATH_INFO: `/v3/users/dc709f9e5c644ab498b746f584670923` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.031178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] RBAC: Authorizing `identity:get_user(user_id=dc709f9e5c644ab498b746f584670923)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.033153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.039688 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 145/289] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/users/dc709f9e5c644ab498b746f584670923 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:59.041158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:59.132840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5345864-347b-419c-941a-461bfafe8928 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.149135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff359dd-7c8a-4f01-826a-e222abfd9880 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.188650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.189577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.190033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.190474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.208073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:59.227748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.228581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.228861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.229139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.235196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.236682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.238309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 145/290] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:58 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:32:59.247373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:59.268389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.270370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.380145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:59.406823 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 146/291] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:32:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:32:59.408999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:59.473727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:59.485176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93a5d8-83f8-43e6-b5b0-0eb3300be167 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.518703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f78ba-d9e7-4f5e-953f-a7941b02f517 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.555109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.556464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.557079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.557553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.558947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.560832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.568278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 147/292] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:32:59 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:32:59.569936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:32:59.611249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.611936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.612216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.612490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/f310e99dafed4118a7064092835b5754/roles/4547e5401a2c4a38b03ce9c68699067d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.623325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, user_id=f310e99dafed4118a7064092835b5754, role_id=4547e5401a2c4a38b03ce9c68699067d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.624815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8454c-7aea-4ab8-9283-b63b7b4d7e9c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.655947 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 146/293] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:32:59 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/users/f310e99dafed4118a7064092835b5754/roles/4547e5401a2c4a38b03ce9c68699067d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:32:59.695124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee83201-7165-4b5c-b8c1-4a69b0148e28 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:32:59.710940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:32:59.734984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:32:59.735967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:32:59.736439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:32:59.736922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:32:59.737844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=dc709f9e5c644ab498b746f584670923) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:32:59.738345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:32:59.740072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6b4d4-fc67-4a83-8417-9dfe7a8cba44 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:32:59.750677 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 148/294] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:32:59 2022] GET /identity/v3/role_assignments?user.id=dc709f9e5c644ab498b746f584670923&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:00.497985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424264-f002-41cf-84cc-b88f057231b1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.498528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424264-f002-41cf-84cc-b88f057231b1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.498606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74424264-f002-41cf-84cc-b88f057231b1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.500219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 147/295] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:00.506241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2abc-faae-4b3d-a31f-91a8772e5a75 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.506887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2abc-faae-4b3d-a31f-91a8772e5a75 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.507366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2abc-faae-4b3d-a31f-91a8772e5a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.527002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-de0b2abc-faae-4b3d-a31f-91a8772e5a75 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:00.528525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de0b2abc-faae-4b3d-a31f-91a8772e5a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:00.604339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 149/296] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:00.894867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c19d7-6feb-407f-be85-29f22c1e9272 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.895583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c19d7-6feb-407f-be85-29f22c1e9272 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.896071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c19d7-6feb-407f-be85-29f22c1e9272 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.897858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 148/297] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:00.903550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.904425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.904902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.908916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbaf7-bd6d-46ea-9b98-3d803617933e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.909202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbaf7-bd6d-46ea-9b98-3d803617933e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.909477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bbaf7-bd6d-46ea-9b98-3d803617933e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.910776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 149/298] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:33:00.915942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:00.916332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:00.916688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:00.927754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:00.928994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:00.936329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:00.937506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:01.010275 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 150/299] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:01.011166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.017080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 150/300] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:01.051242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.123207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd1ad7-a377-4f0a-8f30-9e324c02ab98 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.158676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.159226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.159501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.159769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.170763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:01.175509 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:01.176373 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:01.177143 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 151/301] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:01.184283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.241426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.297398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045ae3e-6597-4836-a65d-4d723acf925f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.334395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.335292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.335733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.336259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.337091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:01.337538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.339090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.347880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 151/302] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:01.353779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.359487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.460454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.466783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7dca2-e0b5-4045-9a59-a82df75ac5cd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.474028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc02f4-d708-4c00-9302-2213ce330152 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.483497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.506829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.507452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.507732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.508542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] PATH_INFO: `/v3/users/b1ccc792f7ff412785419498ee5eb867` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.509689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] RBAC: Authorizing `identity:get_user(user_id=b1ccc792f7ff412785419498ee5eb867)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.511030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.511499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.512337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.512730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 152/303] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/users/b1ccc792f7ff412785419498ee5eb867 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:01.513041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.513509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.513631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.518055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.519791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.523325 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 152/304] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:01.523325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.622025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468386-273f-4d4c-acdc-ab3e2196030a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.640805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12c719-e822-4fc2-929e-fa211a8a2cf4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.665390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.666021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.666301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.666690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.676873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.677199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:01.677522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.677801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.678075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] PATH_INFO: `/v3/system/users/b1ccc792f7ff412785419498ee5eb867/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.679521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b1ccc792f7ff412785419498ee5eb867, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.681537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:01.683036 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:01.683865 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:01.684628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 153/305] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:01.685855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.697816 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 153/306] 10.222.0.52 () {58 vars in 1334 bytes} [Sat Sep 24 20:33:01 2022] PUT /identity/v3/system/users/b1ccc792f7ff412785419498ee5eb867/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:01.698590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.719620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.756665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:01.826715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7780f-e160-4487-8572-c2dcf8a7d435 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.833572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c804-7c1b-451b-b2e8-2219c2df6188 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:01.864667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.865287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.865633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.865836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.866679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:01.867001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.869235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.879212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 154/307] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:01.884558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:01.885552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:01.885817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:01.886381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:01.887211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:01.888094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:01.891291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:01.915839 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 154/308] 10.222.0.52 () {60 vars in 1134 bytes} [Sat Sep 24 20:33:01 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:02.027656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e349b-8a8b-485f-ada4-70e3736d6130 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:02.029735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:02.073104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:02.073750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:02.074031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:02.074309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] PATH_INFO: `/v3/users/7018d78b9ecc44e794be4843c2e4125e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:02.089939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] RBAC: Authorizing `identity:get_user(user_id=7018d78b9ecc44e794be4843c2e4125e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:02.091474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06de856-3986-4685-a3c1-c002c74dcb82 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:02.093595 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 155/309] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:01 2022] GET /identity/v3/users/7018d78b9ecc44e794be4843c2e4125e => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:02.099574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:02.191809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:02.210820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf166-313e-47cf-be6e-ef13bae8a71b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:02.246795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:02.247411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:02.247687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:02.247955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:02.248740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, user.id=7018d78b9ecc44e794be4843c2e4125e, scope.system=all) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:02.249045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:02.250548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27482c9-6c9d-4176-ac6b-42b75c04f55a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:02.256272 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 155/310] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 20:33:02 2022] GET /identity/v3/role_assignments?user.id=7018d78b9ecc44e794be4843c2e4125e&scope.system=all&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:02.332562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e6d7-f0f2-48fd-bafc-239732dce50d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:02.332837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e6d7-f0f2-48fd-bafc-239732dce50d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:02.333115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6e6d7-f0f2-48fd-bafc-239732dce50d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:02.334766 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 156/311] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:02.340917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:02.341225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:02.341537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:02.361578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:02.362816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:02.440948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 156/312] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:02.780559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6e6d7-f0f2-48fd-bafc-239732dce50d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:02.890827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6e6d7-f0f2-48fd-bafc-239732dce50d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:02.926071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:02.926726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:02.927005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:02.927274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:02.937932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:02.942743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:02.943641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:02.944400 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 157/313] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:02.952320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:03.034685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:03.057169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7551746-4bf1-454b-b3e1-bd4d6e27a61e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:03.092440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:03.093051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:03.093334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:03.093603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:03.094265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] RBAC: Adding query filter params (name=reader) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:03.094595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:03.095963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:03.104650 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 157/314] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:03.110106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:03.162420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:03.289750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e52c-495c-4607-b7ad-3ee29b77f7a8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:03.355882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:03.356531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:03.356807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:03.357076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] PATH_INFO: `/v3/users/f310e99dafed4118a7064092835b5754` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:03.368015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] RBAC: Authorizing `identity:get_user(user_id=f310e99dafed4118a7064092835b5754)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:03.369537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:03.371880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 158/315] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:03 2022] GET /identity/v3/users/f310e99dafed4118a7064092835b5754 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:03.377380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:03.533504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abaaac-6945-46c4-b1ab-3503d1f8275f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:03.569778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:03.571618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:03.572126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:03.572600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:03.598887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:03.661681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:03.663916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:03.789858 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 158/316] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:03.795615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:03.948650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:03.962128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2134739-accc-46cb-b3bf-7d37264b2274 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:03.998305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:03.999259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:03.999718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.000169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.001610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:04.003303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6778d46-b853-4578-bc23-8db416ef66cd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:04.005415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 159/317] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:03 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:04.011666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:04.056476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:04.115495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa7a71-1560-4d30-9546-7c7135724e63 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:04.155129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:04.155741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.156021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.156291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.157013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532, user.id=f310e99dafed4118a7064092835b5754) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:04.157317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:04.158773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576e126-6435-4059-8b8b-c3f5d38e1803 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:04.164874 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 159/318] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:04 2022] GET /identity/v3/role_assignments?user.id=f310e99dafed4118a7064092835b5754&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:04.316270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9005-f9ee-4b5d-b080-ae406017da16 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.316564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9005-f9ee-4b5d-b080-ae406017da16 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.316745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9005-f9ee-4b5d-b080-ae406017da16 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.318371 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 160/319] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:04.324936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeb75d-6d3d-4ae8-9355-b57ebb38c978 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.325279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeb75d-6d3d-4ae8-9355-b57ebb38c978 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.325500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeb75d-6d3d-4ae8-9355-b57ebb38c978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.345699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-abeeb75d-6d3d-4ae8-9355-b57ebb38c978 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:04.347025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-abeeb75d-6d3d-4ae8-9355-b57ebb38c978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:04.421580 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 160/320] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:04.690068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055bec-64e1-4a26-b2bd-3f77759028ee None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.690344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055bec-64e1-4a26-b2bd-3f77759028ee None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.690845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055bec-64e1-4a26-b2bd-3f77759028ee None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.692259 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 161/321] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:04.697529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539fb35-15ae-4c2f-9b41-4176acf63c07 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.697824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539fb35-15ae-4c2f-9b41-4176acf63c07 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.698098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539fb35-15ae-4c2f-9b41-4176acf63c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.716316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5539fb35-15ae-4c2f-9b41-4176acf63c07 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:04.717483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5539fb35-15ae-4c2f-9b41-4176acf63c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:04.745026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:04.750489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de055bec-64e1-4a26-b2bd-3f77759028ee None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:04.791536 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 161/322] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:04.795283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed576be5-f585-4b06-bc54-be279f9cfbf6 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.795575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed576be5-f585-4b06-bc54-be279f9cfbf6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.795850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed576be5-f585-4b06-bc54-be279f9cfbf6 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.797116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 162/323] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:33:04.801900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.802189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.802465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.821859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:04.823079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:04.862409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de055bec-64e1-4a26-b2bd-3f77759028ee None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:04.894639 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 163/324] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:04.905752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:04.906395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:04.906740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:04.907021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:04.917944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:04.923306 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:04.924209 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:04.927386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 162/325] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:04.930715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.021087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.030001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04092b0-d765-49e2-8c22-ac0200ecdadb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.062850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.063452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.063728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.063997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.064643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] RBAC: Adding query filter params (name=member) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:05.064946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.066309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.074988 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 164/326] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.080408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.103824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.123697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.196087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f0df2-8616-43f6-a017-f5bbe989870f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.233454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.234080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.234354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.234658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] PATH_INFO: `/v3/users/b1ccc792f7ff412785419498ee5eb867` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.235793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] RBAC: Authorizing `identity:get_user(user_id=b1ccc792f7ff412785419498ee5eb867)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.237339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.240197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 163/327] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/users/b1ccc792f7ff412785419498ee5eb867 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.241038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.249777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91208cc3-aa03-4a5c-b57e-81f8ebfa205b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.284792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.285414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.285690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.285961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.287076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.288506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.290561 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 165/328] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.294649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.316250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.326201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.343807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4710e42-ca87-4247-ba00-92fba6bc97e8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.384068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.384717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.385151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.385589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.393979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e04f8-a951-46d2-9c0a-b514aeae1b04 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.399000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:05.409520 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:05.410685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:05.411644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 164/329] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:05.412907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.434978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.435967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.436243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.436512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.437229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] RBAC: Adding query filter params (role.id=ca0ddc6d9bf24f56b033bf1e448069fb, user.id=b1ccc792f7ff412785419498ee5eb867, scope.system=all) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:05.437534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.438979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.466626 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 166/330] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/role_assignments?user.id=b1ccc792f7ff412785419498ee5eb867&scope.system=all&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.466626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.556973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39475c6-3408-4724-aebc-19ec5699fc20 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.572887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.597057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8db507-0d24-496f-8aea-6e3cf0a1deb8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.606351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.607020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.607299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.607571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.608215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.609707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.639768 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 165/331] 10.222.0.52 () {60 vars in 1134 bytes} [Sat Sep 24 20:33:05 2022] POST /identity/v3/groups => generated 226 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:05.657450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.662532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.674375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 167/332] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.681540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.688023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:05.825105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7919a-ced6-4d57-862d-45d39f9a1218 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:05.866387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:05.867106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:05.867345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:05.867617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] PATH_INFO: `/v3/users/7018d78b9ecc44e794be4843c2e4125e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:05.868772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] RBAC: Authorizing `identity:get_user(user_id=7018d78b9ecc44e794be4843c2e4125e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:05.870246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105978da-3cfe-4fe3-8224-0562ecd60667 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:05.872150 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 166/333] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:05 2022] GET /identity/v3/users/7018d78b9ecc44e794be4843c2e4125e => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:05.878087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:05.995191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc122b-239f-4259-9090-d4ef40300a7c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:06.042193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:06.043188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:06.043188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:06.043488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] PATH_INFO: `/v3/system/users/7018d78b9ecc44e794be4843c2e4125e/roles/4547e5401a2c4a38b03ce9c68699067d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:06.044927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7018d78b9ecc44e794be4843c2e4125e, role_id=4547e5401a2c4a38b03ce9c68699067d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:06.046437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4d876-1d05-4ed4-abd0-798451e61922 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:06.059904 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 168/334] 10.222.0.52 () {58 vars in 1334 bytes} [Sat Sep 24 20:33:05 2022] PUT /identity/v3/system/users/7018d78b9ecc44e794be4843c2e4125e/roles/4547e5401a2c4a38b03ce9c68699067d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:08.977832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775860-e9f1-4a39-a16c-069679ed8164 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:08.978341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775860-e9f1-4a39-a16c-069679ed8164 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:08.978341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59775860-e9f1-4a39-a16c-069679ed8164 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:08.980159 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 167/335] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:08.985686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a54404-195c-4d7d-bc16-76ec211b030f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:08.986242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a54404-195c-4d7d-bc16-76ec211b030f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:08.986242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a54404-195c-4d7d-bc16-76ec211b030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.011929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d3a54404-195c-4d7d-bc16-76ec211b030f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:09.013131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d3a54404-195c-4d7d-bc16-76ec211b030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:09.015173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:09.099938 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 169/336] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:09.119015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b73e7-937a-4374-b217-498b98974787 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.119299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b73e7-937a-4374-b217-498b98974787 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.119752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b73e7-937a-4374-b217-498b98974787 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.121490 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 168/337] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:09.126608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15eb30-d7e4-45d1-84db-822f183f29ec None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.126903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15eb30-d7e4-45d1-84db-822f183f29ec None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.127179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15eb30-d7e4-45d1-84db-822f183f29ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.156112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc15eb30-d7e4-45d1-84db-822f183f29ec None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:09.157319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc15eb30-d7e4-45d1-84db-822f183f29ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:09.186769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed61d82-4bbd-4deb-8096-98d1c4d1f357 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.187423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed61d82-4bbd-4deb-8096-98d1c4d1f357 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.187907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed61d82-4bbd-4deb-8096-98d1c4d1f357 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.189659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 169/338] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:09.195596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.196165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.196663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.231598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:09.233145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:09.261117 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 170/339] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:09.352557 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 170/340] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:09.831862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b70499-6a29-4f3c-af14-922e3ae07a12 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.832574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b70499-6a29-4f3c-af14-922e3ae07a12 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.833069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b70499-6a29-4f3c-af14-922e3ae07a12 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.836305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:09.839077 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 171/341] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:09.839276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:09.841310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:09.841591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:09.841862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:09.879582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:09.880770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:09.993507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.001747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 172/342] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:10.002943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.004270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5b6e7-7735-4752-a60e-44dd907788e2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.048521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.049427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.049840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.050254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.064258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:10.069318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:10.070941 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 20:33:10.072009 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:10.075358 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 171/343] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:10.076343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.134388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a08bf5-65f6-4a96-b3a1-d659322f343c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.162941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.176113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.176996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.177474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.177905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.189405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:10.195787 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:10.196667 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:10.197443 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:10.200377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 173/344] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:10.201945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.246439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73018559-529c-45cd-aa33-2586c3f0e9e0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.318709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.351829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.352486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.352722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.352996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.371628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc96a4b-21f0-47ec-8b33-30ab7c412e03 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.395285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:10.401038 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:10.402000 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:10.402936 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:10.412310 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 172/345] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:10.414069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.429998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.429998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.430351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.430422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.431074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] RBAC: Adding query filter params (name=reader) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:10.431381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:10.432740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10eeda-7d0a-4a08-bf7d-67d233334194 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:10.446935 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 174/346] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:10.448612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a924b2d-3927-414f-b9fc-ad2afe61a83a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.448897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a924b2d-3927-414f-b9fc-ad2afe61a83a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.449172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a924b2d-3927-414f-b9fc-ad2afe61a83a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.452262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 175/347] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:10.452262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a924b2d-3927-414f-b9fc-ad2afe61a83a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.564443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aea6e-2121-4409-ae62-c2863cc81338 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.592972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.593231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a924b2d-3927-414f-b9fc-ad2afe61a83a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.609930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.610939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.611377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.611782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.612763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:10.613595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:10.615591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:10.619779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.626536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 173/348] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:10.627545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.638320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.640837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.640837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.640837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.640837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:10.640837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:10.642067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95039ca6-f5cb-44b2-8e79-20104a72ac55 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:10.651621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 176/349] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:10.653211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.653656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.653930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.677451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:10.679020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:10.768667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fe0b0-aabe-4763-a3e5-5c4a14503243 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.781226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 177/350] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:10.782284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.810777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.811417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.811695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.811969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] PATH_INFO: `/v3/users/7018d78b9ecc44e794be4843c2e4125e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.813102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] RBAC: Authorizing `identity:get_user(user_id=7018d78b9ecc44e794be4843c2e4125e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:10.814737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:10.816396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 174/351] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/users/7018d78b9ecc44e794be4843c2e4125e => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:10.817073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:10.884312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.936452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5320ec-7fd4-4dfa-9037-96be494c21cb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.947861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791be22-668e-45b3-bcd0-3cc4017d557c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:10.954995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:10.980999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.981982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.982271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.982611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.992861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:10.993468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:10.993754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:10.994029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:10.997094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:11.003965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:11.005012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:11.005907 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:11.010895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 178/352] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:11.010895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.094385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.099542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.144629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d795d-de6c-47e8-a803-41fea571a788 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.162537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.185178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.185780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.186049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.186317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.221909 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 175/353] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:11.222690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.234658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.266684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.268154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c8847f-bae0-428c-befa-07187092afcc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.343230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005da3e-89eb-42d9-b0ed-7d6cc03d583d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.386187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.387515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.387515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.387515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.388005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] RBAC: Adding query filter params (role.id=4547e5401a2c4a38b03ce9c68699067d, user.id=7018d78b9ecc44e794be4843c2e4125e, scope.system=all) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:11.388308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.389775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.393856 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 179/354] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:11.394807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232dc002-ee20-4767-89e6-0c70bf3749b0 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.394917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232dc002-ee20-4767-89e6-0c70bf3749b0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.395169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232dc002-ee20-4767-89e6-0c70bf3749b0 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.396675 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 180/355] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:11.397393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232dc002-ee20-4767-89e6-0c70bf3749b0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.404072 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 176/356] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/role_assignments?user.id=7018d78b9ecc44e794be4843c2e4125e&scope.system=all&role.id=4547e5401a2c4a38b03ce9c68699067d => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:11.405218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.419569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.506226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.545772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338b69c-e9c2-456d-8638-9489f60cf960 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.556475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232dc002-ee20-4767-89e6-0c70bf3749b0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.593228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.593891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.594860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.595212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.602463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.603113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.603385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.603674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.604327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] RBAC: Adding query filter params (name=member) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:11.605432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.606018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.608949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:11.614389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:11.615409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:11.616385 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 177/357] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:11.618259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.623665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 181/358] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:11.624403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.713119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.748990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42267cc-db12-4800-87d9-8db434ad3d1f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.759685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befdbb8-2829-4f35-9d99-0f4c18747591 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.787780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.787780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.787780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.787780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.789043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.790531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ad222-8cbc-4a97-bfe9-f300db7c7525 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.792157 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 178/359] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:11.793408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.793686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.793962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.801054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.801695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.801974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.802242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.803963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.805112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:11.807438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 182/360] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:11.807541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.810227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.814763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:11.815983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:11.890741 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 179/361] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:11.891799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:11.945261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137745ae-415e-4f57-ab55-600fb57f119b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:11.973993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:11.990429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:11.991072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:11.991348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:11.991712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:11.992373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:11.992673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:11.994029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.006314 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 183/362] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.008643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.028438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab1a5-4633-4b3d-a5e8-b225e4318a5f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.073338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.073955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.074227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.074513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.084226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.131108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a1204e-9484-4d93-9d36-1c53938e8e5a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.160371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.161766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.175121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.176029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.176510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.177257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.186317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.188050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.190418 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 184/363] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.192404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.243627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.284449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 180/364] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:12.285321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.320737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c35d-3a48-464f-ad50-496b4f950483 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.348490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.363693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.365444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.365719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.365988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.368816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.370326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.372237 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 185/365] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.377160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.403459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af47706-4d26-4972-bedf-ad42b9bfbe4d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.449905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.450187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.450459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.450754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.486269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eee72-2e21-4b51-8f1f-8d18237d9223 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.488179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.520552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.521145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.521422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.521684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.521911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.522385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=b64a55cff23746df99bee214696e0d1e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:12.522857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.523483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.524617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.530538 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 186/366] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.531309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.596943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.646567 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 181/367] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:12.650824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d38090-5a2c-4575-bbec-f5d0ee254508 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.652694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.692073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.692715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.692992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.693262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.705626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:12.711189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:12.712109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:12.712903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 187/368] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:12.713569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.729960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.768681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa2b68-e437-436b-b9a0-714fd088a01c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.805811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.806570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.806848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.807112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.808223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.809797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.811718 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 182/369] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.816813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.825428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf23562-c038-4ee8-b275-e6abb241f001 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:12.845678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:12.860707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:12.861315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:12.861591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:12.861860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:12.862636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=b64a55cff23746df99bee214696e0d1e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:12.862943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:12.864366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:12.870211 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 188/370] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=2413f08d373842418dcfc2305d420a05 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:12.871024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:12.957701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dc03d-9e22-4672-84bd-6534edf29dcd None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.005211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.005598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76361736-bd21-4cc1-85c2-1eeed0f1d69d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.006092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.006469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.006864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.008936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.010844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.017566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 183/371] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.018674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.023349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:13.051310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.052608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.053027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.053532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.055133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:13.055708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.058118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.068170 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 189/372] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.069269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.131004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:13.144917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148e01-b40d-40ec-9bf5-93ae888c86bc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.181855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.183327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.183776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.184216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.185722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.187453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.189274 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 184/373] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.195456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.196861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3436c33-e7dd-4410-b310-31f9936ec7c5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.237215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.238125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.238611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.239067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.240420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.242250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.244376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 190/374] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.245507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.287049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:13.327497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f34cb-7cd9-41d4-b017-8445f15964f4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.362406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdd473-1ada-4733-9f30-4bb65ee2829c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.369162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.370217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.370794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.371256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.378150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:13.404745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.405356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.405761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.406163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.406926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:13.407237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.409341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.422747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 191/375] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.422747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.507423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.507423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.609825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:13.674733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 185/376] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:13.683310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ab17-0866-4875-bcb5-77c1bb70cf6b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.689889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.737760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.739903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.741330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.742116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.743377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:13.744040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.747115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.760665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 192/377] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=2413f08d373842418dcfc2305d420a05 => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.863033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e38116-f5ba-4db6-a640-b8656e334b3c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:13.911024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:13.911954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:13.912402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:13.912861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] PATH_INFO: `/v3/projects/a8f45f34e55f4f50b20d34824a4e60af` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:13.915067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] RBAC: Authorizing `identity:get_project(project_id=a8f45f34e55f4f50b20d34824a4e60af)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:13.916697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:13.918581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 186/378] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/projects/a8f45f34e55f4f50b20d34824a4e60af => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:13.924510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:13.982000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:14.061353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a64b1-c574-47e9-84ba-339e3b62ae0a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:14.107899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:14.108550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:14.108824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:14.109092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] PATH_INFO: `/v3/groups/7b8c002e30c248d78aee73fde2f6d7da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:14.118387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] RBAC: Authorizing `identity:get_group(group_id=7b8c002e30c248d78aee73fde2f6d7da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:14.119957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ec5ae-41a5-49b0-8d1e-f6a2cec37578 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:14.121949 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 193/379] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:13 2022] GET /identity/v3/groups/7b8c002e30c248d78aee73fde2f6d7da => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:14.127808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:14.173747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:14.242678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fc22d-c668-42e1-8e4a-1189af5daa52 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:14.296086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:14.296712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:14.296988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:14.297264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:14.297988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] RBAC: Adding query filter params (group.id=7b8c002e30c248d78aee73fde2f6d7da, role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=a8f45f34e55f4f50b20d34824a4e60af) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:14.298294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:14.299824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983892da-a6cc-4fe6-ab1c-2b5d21aa46b4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:14.310935 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 187/380] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:14 2022] GET /identity/v3/role_assignments?group.id=7b8c002e30c248d78aee73fde2f6d7da&scope.project.id=a8f45f34e55f4f50b20d34824a4e60af&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:15.337468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30d79-09c0-45e3-9105-50bff503f242 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:15.338010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30d79-09c0-45e3-9105-50bff503f242 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:15.338010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a30d79-09c0-45e3-9105-50bff503f242 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:15.339622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 194/381] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:15.352729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306228-6474-45ed-a63b-1db111ce36fd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:15.353215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306228-6474-45ed-a63b-1db111ce36fd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:15.353318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306228-6474-45ed-a63b-1db111ce36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:15.375102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-15306228-6474-45ed-a63b-1db111ce36fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:15.376273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15306228-6474-45ed-a63b-1db111ce36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:15.471393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 188/382] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:15.607945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfec69d-5118-4b2f-bd78-04afedb4070b None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:15.608560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfec69d-5118-4b2f-bd78-04afedb4070b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:15.609029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfec69d-5118-4b2f-bd78-04afedb4070b None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:15.610816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 195/383] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:15.616072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:15.616807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:15.617460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:15.637947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:15.640325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:15.684274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:15.733964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 189/384] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:15.833919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfec69d-5118-4b2f-bd78-04afedb4070b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:15.980282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfec69d-5118-4b2f-bd78-04afedb4070b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:15.988466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:16.030142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.030802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.031079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.031422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.042058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:16.050939 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:16.051824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:16.052591 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 196/385] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:16.059575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.195397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ef9ba-87d2-49a8-a281-c7fa00846d4d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.249773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.250303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.250758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.251040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.251702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:16.252002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:16.253561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82054-80df-4952-a57a-b174dea69ff8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:16.266452 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 190/386] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:16.271799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.338725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb0c7-9861-41b0-b5e4-432ebf5df92a None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.339047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb0c7-9861-41b0-b5e4-432ebf5df92a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.339281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edb0c7-9861-41b0-b5e4-432ebf5df92a None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.340865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 191/387] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:16.345681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.345965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.346280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.368521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:16.369717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:16.405860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82e877-a8b9-4136-b0bf-d4f562f33170 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.413174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:16.441276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.442107 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 192/388] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:16.443103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.446809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.447116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.447393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.448606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:16.450053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:16.455327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 197/389] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:16.461837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.534947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:16.570555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4155cc-146d-4dc5-8fb9-c67f1cc7a7f9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.653182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.653834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.654108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.654374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.679853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:16.685176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:16.686043 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:16.686899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:16.689143 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 193/390] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:16.693114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870df89c-d5bb-4f19-9fbd-4d3ec525bd46 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.696248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.735441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:16.763889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.764525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.764799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.765063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.766612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:16.768038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:16.775945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 198/391] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:16.776757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.870724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca6f00-7ddb-4880-b3e6-7cba69e6bb9b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.903750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b313b1-4ef3-404e-96d5-5add701a12c2 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:16.905627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:16.915769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.916371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.916646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.917268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.917938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:16.918235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:16.920221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:16.928538 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 194/392] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:16.933624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:16.952213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:16.952873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:16.953148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:16.953474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/groups/cab40f60f9604ab2b38023eee092c8c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:16.956962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, group_id=cab40f60f9604ab2b38023eee092c8c0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:16.958409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:16.976250 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 199/393] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 20:33:16 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/groups/cab40f60f9604ab2b38023eee092c8c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:16.977550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.099344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.105759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb8da0-9359-44c8-b6ef-8299d1ba421a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.150042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.151016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.151453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.151868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.153407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.155164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.156504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf9495-7470-4203-b258-5e42028efe0c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.157457 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 200/394] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:17.162931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.217459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.218101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.218375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.218677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.218919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.232504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8c6db7cf-a7f3-4998-a42b-f99bb0f9328e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:17.237807 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:17.238740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:17.239601 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 195/395] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:17.240237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4301df-1d94-4a62-9999-acd1578027a1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.241035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4301df-1d94-4a62-9999-acd1578027a1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.241035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4301df-1d94-4a62-9999-acd1578027a1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.246201 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 196/396] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:17.247354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4301df-1d94-4a62-9999-acd1578027a1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.316061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba937fac-de37-492c-8a26-f9ae175dc657 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.354380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.356669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.357036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.357307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.358126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4301df-1d94-4a62-9999-acd1578027a1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.358422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.360005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d4dfd-ce71-4e17-90e6-0752157d195e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.367101 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 201/397] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:17 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:17.367101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.367101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.367101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.385050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:17.386242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:17.395153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.395817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.396099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.396371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.397029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:17.397863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.399920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23554657-0fcc-4aeb-9761-6941afb676de None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.406800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.411704 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 197/398] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:17.412952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe3b0-f794-44d0-b448-ffd92abb25d9 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.413625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe3b0-f794-44d0-b448-ffd92abb25d9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.413898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe3b0-f794-44d0-b448-ffd92abb25d9 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.419093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 198/399] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:17.419093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffe3b0-f794-44d0-b448-ffd92abb25d9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.471299 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 202/400] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:17.483419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.514734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.538802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffe3b0-f794-44d0-b448-ffd92abb25d9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.587483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.589077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.589592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.590063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e/groups/cab40f60f9604ab2b38023eee092c8c0/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.593553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] RBAC: Authorizing `identity:create_grant(project_id=b64a55cff23746df99bee214696e0d1e, group_id=cab40f60f9604ab2b38023eee092c8c0, role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.595838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271ec18-f87f-48a2-ae5e-0efbc42e9b4b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.597494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed4d18-436b-4c13-b50b-195c60ff67aa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.619713 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 199/401] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 20:33:17 2022] PUT /identity/v3/projects/b64a55cff23746df99bee214696e0d1e/groups/cab40f60f9604ab2b38023eee092c8c0/roles/2413f08d373842418dcfc2305d420a05 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:17.619713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.619713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.619713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.656291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:17.661674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:17.668719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.669511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.669827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.670105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.674033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.675543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.683269 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 203/402] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:17 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:17.684277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.788596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.816226 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 200/403] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:17.817086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.857957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:17.868394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a8fe-fd27-4bfa-8c88-c9989117f82e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:17.942064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:17.942643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:17.942924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:17.943197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:17.944960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:17.946735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:17.948571 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 204/404] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:17 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:17.954157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:17.974759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53aa4c-28a2-4ce7-9424-a5e80bb68ae3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.018382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.019070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.019348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.019614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.031164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:18.035962 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:18.036867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:18.037627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:18.042373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 201/405] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:18.044039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.083772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14abe6b-abc7-45aa-8b41-31295e0064b3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.099992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:18.138950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.139624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.139902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.140168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/groups/cab40f60f9604ab2b38023eee092c8c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.143108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, group_id=cab40f60f9604ab2b38023eee092c8c0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.144557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.167662 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 205/406] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 20:33:17 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/groups/cab40f60f9604ab2b38023eee092c8c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:33:18.185552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aa2b4-caf4-4e15-8cd2-fc443331d9de None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.212026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.240675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:18.252358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.252997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.253272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.253543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.254229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:18.254796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.256427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.274631 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 202/407] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:18.279970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.326620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6473672-3a46-4455-8abb-6b4dca58d5ee None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.368517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.369489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.370066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.370529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.382637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:18.388211 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:18.389113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:18.389872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:18.392468 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 206/408] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:18.400086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.511063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:18.520871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6779ee3e-58b5-43d3-a0f8-d226a504df31 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.561178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.562695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.563218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.563647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.564553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:18.565157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.566760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.567790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66336eca-1b9b-4903-9341-6564d6f16c1e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.580416 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 207/409] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:18.587377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.589640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:18.614378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.615268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.615431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.615713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.616873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.618388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.626308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 203/410] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:18.626899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.698251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf7d0a-5a0b-4f3d-baf5-a1f9cbf0b330 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.734068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.734969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.735385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.735790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] PATH_INFO: `/v3/groups/7b8c002e30c248d78aee73fde2f6d7da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.737041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] RBAC: Authorizing `identity:get_group(group_id=7b8c002e30c248d78aee73fde2f6d7da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.738710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.740743 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 208/411] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/groups/7b8c002e30c248d78aee73fde2f6d7da => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:18.747116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.765144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:18.847557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de042ce-8b6a-4095-bff9-9d338efcc034 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.854190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05947ec-5c8d-4374-8907-572cad384121 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:18.892015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.893056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.893476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.893885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] PATH_INFO: `/v3/projects/a8f45f34e55f4f50b20d34824a4e60af` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.895188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] RBAC: Authorizing `identity:get_project(project_id=a8f45f34e55f4f50b20d34824a4e60af)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.896753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.904631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 209/412] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/projects/a8f45f34e55f4f50b20d34824a4e60af => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:18.906058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:18.928570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:18.929274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:18.929549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:18.929817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:18.935198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:18.936679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:18.938625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 204/413] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:18 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:18.944926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:19.000765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:19.014668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed891988-6331-430a-ac6c-7024c79270bd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:19.053975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:19.054798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:19.055414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:19.055768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] PATH_INFO: `/v3/projects/a8f45f34e55f4f50b20d34824a4e60af/groups/7b8c002e30c248d78aee73fde2f6d7da/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:19.060310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8f45f34e55f4f50b20d34824a4e60af, group_id=7b8c002e30c248d78aee73fde2f6d7da, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:19.061771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fec481-8631-4839-ba86-96db1eb901c4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:19.072459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:19.080296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 210/414] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 20:33:18 2022] PUT /identity/v3/projects/a8f45f34e55f4f50b20d34824a4e60af/groups/7b8c002e30c248d78aee73fde2f6d7da/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:19.186537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cc5bb-922b-4961-a76e-04dd7463765a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:19.242015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:19.242734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:19.243014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:19.243290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/groups/cab40f60f9604ab2b38023eee092c8c0/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:19.252082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8a08d80fc14d0a9c07e248d0c08532, group_id=cab40f60f9604ab2b38023eee092c8c0, role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:19.253579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e0da-7c6c-486d-bcbf-d1cdef0d4566 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:19.271747 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 205/415] 10.222.0.52 () {58 vars in 1442 bytes} [Sat Sep 24 20:33:18 2022] PUT /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532/groups/cab40f60f9604ab2b38023eee092c8c0/roles/2413f08d373842418dcfc2305d420a05 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:33:19.628725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e471d5-2be4-4380-a766-3370aa867857 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:19.629426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e471d5-2be4-4380-a766-3370aa867857 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:19.629426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e471d5-2be4-4380-a766-3370aa867857 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:19.630885 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 211/416] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:19.637254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:19.637613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:19.637830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:19.663839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:19.665359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:19.746854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 206/417] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:20.196667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e471d5-2be4-4380-a766-3370aa867857 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:20.305832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e471d5-2be4-4380-a766-3370aa867857 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:20.342807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:20.343451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:20.343712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:20.343981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:20.355740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:20.360761 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:20.361664 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:20.362434 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 212/418] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:20.369058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:20.413773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:20.488666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632136-abf1-4fe7-9459-83302ef43e85 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:20.528118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:20.528722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:20.528999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:20.529265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:20.529917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:20.530214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:20.531615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:20.540660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 207/419] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:20.546222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:20.550798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:20.653756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a86cc-1e18-4aa7-83b8-ec77193d25b9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:20.689178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:20.689779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:20.690056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:20.690317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:20.763198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:20.764576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7c15-9dcf-4738-9dea-7ad5dacce6b2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:20.766456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:20.870255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 213/420] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:20.875957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:20.908674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102d79-61c5-4c23-b446-0d12a8c5565a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:20.909002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102d79-61c5-4c23-b446-0d12a8c5565a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:20.909230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6102d79-61c5-4c23-b446-0d12a8c5565a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:20.911056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 214/421] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:20.921028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:20.921476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:20.921758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:20.953604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:20.955115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:21.067114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae186d46-d5e0-40c9-abe5-75b7a32ef227 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:21.084284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:21.110529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:21.111321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.111673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.112014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.113232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:21.114887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:21.117810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 208/422] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:20 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:21.123347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:21.160353 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 215/423] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:21.252226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2914ce-cb19-49b3-b09d-cc000c409dee None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:21.304591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:21.305589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.306066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.306555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.307865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:21.309546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76b198-ca2d-4b1d-9248-56e1398306a7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:21.311347 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 209/424] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:21 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:21.317340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:21.365375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:21.463549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c81c-23c4-4ef7-a3bd-32d499f47747 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.463878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c81c-23c4-4ef7-a3bd-32d499f47747 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.464154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c81c-23c4-4ef7-a3bd-32d499f47747 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.465723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 210/425] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:21.470996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.471272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.471537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.483656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07351c1-cee9-4ee5-97cd-f43e7d42a57f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:21.507102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:21.508316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:21.551271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:21.551888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.552162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.552429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.553151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=b64a55cff23746df99bee214696e0d1e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:21.553515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:21.555016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:21.571784 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 216/426] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:21 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:21.575140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:21.603225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:21.680571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 211/427] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:21.781546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:21.792725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8922c-9e8e-48d7-8aac-37d236acf591 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:21.840821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:21.841463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:21.841737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:21.842002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:21.856526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:21.862251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:21.863409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:21.864230 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:21.867673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 217/428] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:21.878608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:21.997202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfee1c7-8213-432a-9319-2ec9ece478a3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.037961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.038763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.039041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.039309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.039973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:22.040339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.041713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.052585 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 212/429] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:22.058901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.153245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.175184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32481c-6b03-4edc-ad0d-34d4d141f02b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.200007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.220984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.222044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.222044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.222294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.256854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.293106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e51a3-2aba-46bd-ae18-95517814fc13 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.310845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.312339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691da2d6-80e4-4b4a-818a-a28d96427e58 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.348742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.349382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.349661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.352362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] PATH_INFO: `/v3/roles/member` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.364035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a47ae8df-2bc9-4bdc-8881-6f54b22b2539 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:22.370002 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:22.371014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 20:33:22.371780 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:22.372387 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 213/430] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:22.374537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916266-e602-4f5b-8939-16f295efd82a None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.374537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916266-e602-4f5b-8939-16f295efd82a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.374537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7916266-e602-4f5b-8939-16f295efd82a None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.375305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 214/431] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:22.376320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe488-08ce-40e5-9aa5-b08fcafb0456 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.376598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe488-08ce-40e5-9aa5-b08fcafb0456 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.376866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe488-08ce-40e5-9aa5-b08fcafb0456 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.378059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 215/432] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:22.378771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbe488-08ce-40e5-9aa5-b08fcafb0456 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.436920 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 218/433] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:22.438157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371839-8e63-4fd3-8d85-201961134a42 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.438448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371839-8e63-4fd3-8d85-201961134a42 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.438756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371839-8e63-4fd3-8d85-201961134a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.456767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aa371839-8e63-4fd3-8d85-201961134a42 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:22.457958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa371839-8e63-4fd3-8d85-201961134a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:22.474640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.488108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbe488-08ce-40e5-9aa5-b08fcafb0456 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.509044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.522640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.523190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.523727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.524000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.524666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] RBAC: Adding query filter params (name=member) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:22.524967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.525686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 219/434] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:22.526350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.526920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.527201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.527470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.536002 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 216/435] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:22.539385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.551500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:22.552689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:22.623737 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 220/436] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:22.627361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.655595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04311823-16cd-48ee-906d-f34672a5da25 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.692180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.693102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.693579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.694635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] PATH_INFO: `/v3/projects/b64a55cff23746df99bee214696e0d1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.696234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] RBAC: Authorizing `identity:get_project(project_id=b64a55cff23746df99bee214696e0d1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.697887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.700192 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 217/437] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/projects/b64a55cff23746df99bee214696e0d1e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:22.705149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.725054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.761434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0bfaa-89ff-4097-864a-5ca5c8bb0e7b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.781014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.801627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.803748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.804283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.804810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.825025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64da7-7cab-4946-a27f-d0bccfd9ce57 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.865428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:22.867206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:22.867483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:22.867776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:22.868891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.870366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.872118 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 218/438] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:22.875609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:22.883596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:22.885218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:22.960300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:22.976527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e29ba-1ef8-4ee9-8947-07fcd5946e6f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:22.989974 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 221/439] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:22.991608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.010238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.011083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.011621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.012006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.012348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.022774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:23.027697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:23.028776 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 219/440] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:23.029566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.093626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bedd1a-8ea3-453b-a109-b4db44275b3a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.126238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.126859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.127137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.127459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.127548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c45780-97a8-4b70-a209-be01fc0d5c52 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.128169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=b64a55cff23746df99bee214696e0d1e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:23.128471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.129875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.136165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 222/441] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:22 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=b64a55cff23746df99bee214696e0d1e&role.id=2413f08d373842418dcfc2305d420a05 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.136926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.169893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.170567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.170845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.171115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.183722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:23.188665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:23.189568 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:23.190339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 220/442] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:23.191031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.204760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.251717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.275907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adbb99-a388-4b27-85ac-ed02ef935017 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.306661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9338db-5505-4624-85eb-44cc67793e87 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.314691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.315519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.315654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.315878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.317546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.318819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.325498 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 223/443] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.326387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.345989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.346601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.346878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.347140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.347930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:23.348230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.349596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.358993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 221/444] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.360260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.440121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.452472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d629200-cf11-4ab0-8621-61b1e3649e77 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.483588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bd5e-a9b0-44fa-a1b2-530c56f03433 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.499768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.500588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.500869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.501141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.501972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:23.502530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.504111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.516060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 224/445] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.516987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.527586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.528224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.528502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.528777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.529978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.533005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.533112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.534196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 222/446] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.535514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.627099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827085-d1a2-42f7-9852-64af97cee7ba None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.646215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f37ad-fc30-4458-9ab6-455a6aa66a41 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.662993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.663603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.663885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.664153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.681110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.681712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.682264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.682570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.685872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.732723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.734189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.745069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.748241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.749695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.835479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 225/447] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:23.836321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.850662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 223/448] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:23.851839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:23.885524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.938399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d76bc-db67-4288-87d3-a19a1437a9f3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.948760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:23.957855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4dacc-c103-4e34-b356-282b92dc4b6e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:23.972149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:23.972752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:23.973037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:23.973311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:23.974029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=ca0ddc6d9bf24f56b033bf1e448069fb, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:23.974339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:23.976337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:23.981908 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 226/449] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=ca0ddc6d9bf24f56b033bf1e448069fb => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:23.982805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:24.004634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.005330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.005561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.005836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] PATH_INFO: `/v3/projects/a8f45f34e55f4f50b20d34824a4e60af` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.007098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=a8f45f34e55f4f50b20d34824a4e60af)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.008603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.010647 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 224/450] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/projects/a8f45f34e55f4f50b20d34824a4e60af => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:24.016201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:24.103577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35740669-19bb-4c92-b75b-26cb5fd35641 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:24.125864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c19b-9244-41d5-9f98-f6e26eaac4f4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:24.129365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:24.141714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.142351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.142654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.142984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] PATH_INFO: `/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.144089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8a08d80fc14d0a9c07e248d0c08532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.145522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.147983 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 227/451] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:33:23 2022] GET /identity/v3/projects/1a8a08d80fc14d0a9c07e248d0c08532 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:24.153813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:24.168026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.168676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.168958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.169232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] PATH_INFO: `/v3/groups/7b8c002e30c248d78aee73fde2f6d7da` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.170434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] RBAC: Authorizing `identity:get_group(group_id=7b8c002e30c248d78aee73fde2f6d7da)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.171951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.173578 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 225/452] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:24 2022] GET /identity/v3/groups/7b8c002e30c248d78aee73fde2f6d7da => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:24.179861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:24.211091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:24.285112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360e39-0874-4b31-ab12-fbcecd29aa13 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:24.309696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03432-3275-42fd-9f18-d51270ebc8c1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:24.325433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.328734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.328734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.328734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] PATH_INFO: `/v3/groups/cab40f60f9604ab2b38023eee092c8c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.328734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] RBAC: Authorizing `identity:get_group(group_id=cab40f60f9604ab2b38023eee092c8c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.329057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.330691 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 228/453] 10.222.0.52 () {56 vars in 1180 bytes} [Sat Sep 24 20:33:24 2022] GET /identity/v3/groups/cab40f60f9604ab2b38023eee092c8c0 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:24.337025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:24.348862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.349574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.349845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.350186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.351117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] RBAC: Adding query filter params (group.id=7b8c002e30c248d78aee73fde2f6d7da, role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=a8f45f34e55f4f50b20d34824a4e60af) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:24.354340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.354340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e49a0-f5a8-4114-bf71-006b8dfd99c7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.358857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 226/454] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:24 2022] GET /identity/v3/role_assignments?group.id=7b8c002e30c248d78aee73fde2f6d7da&scope.project.id=a8f45f34e55f4f50b20d34824a4e60af&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:24.456829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236f0e-f12d-4e45-a25a-6649112e380d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:24.467892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:24.495049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:24.495676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:24.495957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:24.496231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:24.496966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] RBAC: Adding query filter params (group.id=cab40f60f9604ab2b38023eee092c8c0, role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:24.497277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:24.498773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643bb91d-5652-4139-a0c4-4dc4db2eb8a3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:24.505259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 229/455] 10.222.0.52 () {56 vars in 1376 bytes} [Sat Sep 24 20:33:24 2022] GET /identity/v3/role_assignments?group.id=cab40f60f9604ab2b38023eee092c8c0&scope.project.id=1a8a08d80fc14d0a9c07e248d0c08532&role.id=2413f08d373842418dcfc2305d420a05 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:27.456551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f2ae-1151-4fca-adf5-b884be65f724 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:27.457072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f2ae-1151-4fca-adf5-b884be65f724 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:27.457072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28f2ae-1151-4fca-adf5-b884be65f724 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:27.458747 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 227/456] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:27.464202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af893fb-cb84-4a46-94d5-320e1abcd40e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:27.464609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af893fb-cb84-4a46-94d5-320e1abcd40e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:27.464750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af893fb-cb84-4a46-94d5-320e1abcd40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:27.491507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7af893fb-cb84-4a46-94d5-320e1abcd40e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:27.492969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7af893fb-cb84-4a46-94d5-320e1abcd40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:27.580673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 230/457] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:27.766343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c60547-efd9-4bd7-818e-9fd252d46e9b None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:27.766872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c60547-efd9-4bd7-818e-9fd252d46e9b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:27.767319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c60547-efd9-4bd7-818e-9fd252d46e9b None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:27.769108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 228/458] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:27.775044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:27.775502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:27.775921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:27.803691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:27.805158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:27.905067 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 231/459] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:28.073119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c60547-efd9-4bd7-818e-9fd252d46e9b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:28.120365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:28.213394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c60547-efd9-4bd7-818e-9fd252d46e9b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:28.257707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:28.258405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.258712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.258983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.273591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:28.275145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:28.295953 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:28.296867 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:28.297707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:28.297707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:28.297707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:28.297707 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 229/460] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:28.307607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:28.357953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:28.374851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:28.463211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340e4b5-ce50-4406-a6d8-7c3c7355efc6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:28.477899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:28.491284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1d94e-d890-422c-84a8-40aa8296e906 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:28.507783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:28.508518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.508797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.509068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.509710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:28.510015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:28.511426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd28ed-f3a3-483e-b502-4bb8c7827a5b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:28.525849 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 232/461] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:28.527865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e232c0-5249-41b1-902f-162d89980140 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.528548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e232c0-5249-41b1-902f-162d89980140 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.529137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e232c0-5249-41b1-902f-162d89980140 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.529989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:28.530726 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 233/462] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:28.531614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada834-031b-4169-b932-941565f91ab0 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.531895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada834-031b-4169-b932-941565f91ab0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.532167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada834-031b-4169-b932-941565f91ab0 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.533487 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 234/463] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:28.534692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.534916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.535198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.538689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada834-031b-4169-b932-941565f91ab0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:28.547712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:28.549590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-7ee2f27f-bf91-4b82-8b7e-4ee542585756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:28.564640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:28.565554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:28.566389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:28.566389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:28.566389 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:28.566389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 230/464] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:28.567200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43daf4-8b80-4bfe-9edd-515d5f15ed3d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.567200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43daf4-8b80-4bfe-9edd-515d5f15ed3d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.567489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43daf4-8b80-4bfe-9edd-515d5f15ed3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.587370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ab43daf4-8b80-4bfe-9edd-515d5f15ed3d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:28.588554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ab43daf4-8b80-4bfe-9edd-515d5f15ed3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:28.655360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:28.665470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fada834-031b-4169-b932-941565f91ab0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:28.666896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 231/465] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:28.668554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.669001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.670397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.693895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:28.695652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:28.709931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:28.710909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.711095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.711322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.711951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:28.713971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:28.747056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 235/466] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:28 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:28.748132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:28.778147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:28.783700 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 232/467] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:28.896460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4904c1-0697-4b93-b07c-25df55e147a1 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:28.934487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:28.935187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:28.935462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:28.935739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:28.936392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:28.936692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:28.938061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:28.952198 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 236/468] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:28.957635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.008951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.028252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:29.085901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ff41-0f3b-4f0e-876c-261f7d361f73 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.130225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.131755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.132441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.133861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.134833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.136812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.172654 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 233/469] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:28 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:29.189561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40960-cae6-4a6b-9672-6f230d0e6dad None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.193215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.245815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.246478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.246826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.247095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.262469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.263996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.272333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c7e48b4b-7660-4187-8742-0c3843613fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:29.283563 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:29.284497 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:29.285923 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:29.285923 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:29.285923 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:29.285923 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 237/470] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:29.286788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583261-4f76-46af-a0a2-5bfc9cd9367d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.288095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583261-4f76-46af-a0a2-5bfc9cd9367d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.288357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75583261-4f76-46af-a0a2-5bfc9cd9367d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.290040 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 238/471] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:29.294561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583261-4f76-46af-a0a2-5bfc9cd9367d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.342934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacf175-5529-4e79-b735-9a7ccd13ca09 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.377657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:29.387944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.388597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.388935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.389312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.402690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.404321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4e50d728-db27-4906-aca1-e63a0fe2095f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:29.422558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:29.423556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:29.424420 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:29.424420 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:29.424420 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:29.424420 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 234/472] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:29.429333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.429333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.429333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.429610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75583261-4f76-46af-a0a2-5bfc9cd9367d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.449715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:29.451204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:29.471179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.471840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.472118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.472384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.473011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:29.473307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.475363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.493214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 239/473] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:29.494970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.549374 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 235/474] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:29.551381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.645305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:29.650922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87264864-ff37-477d-a1b3-554a4e833109 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.695733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:29.720100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.720717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.720997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.721270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.721907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:29.722207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.723601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.744377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183155-04da-4014-af8b-e08d5e85cb98 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.745484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 240/475] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:29.755458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:29.793995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.794647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.794928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.795204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.795851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.797295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.837738 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 236/476] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:29 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:29.922197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be575-57d0-46e4-8001-50d923d1323e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:29.972751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:29.973720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:29.974202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:29.974740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:29.975636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:29.977331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:29.997068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:30.015014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 241/477] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:29 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:30.139744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:30.148721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303382-5463-4b3e-8a81-a8513f8864dc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:30.187677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:30.188697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:30.189180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:30.189654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:30.190632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:30.192265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:30.203733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.sql.core [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 20:33:30.203733 ncdv-master-1 devstack@keystone.service[99133]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 20:33:30.203733 ncdv-master-1 devstack@keystone.service[99133]: [parameters: {'id': 'e9b094c0f8d34f9ca2f0217995ec4a6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 20:33:30.203733 ncdv-master-1 devstack@keystone.service[99133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 20:33:30.222332 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return ref.to_dict() Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:33:30.223462 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application yield resource Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise self.value Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application session.commit() Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 20:33:30.224397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.flush() Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application compat.raise_( Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 20:33:30.225311 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:33:30.226164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:33:30.227006 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [parameters: {'id': 'e9b094c0f8d34f9ca2f0217995ec4a6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:30.227795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 20:33:30.228559 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:30.232475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 237/478] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 20:33:29 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:30.238597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:30.284575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:30.353747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535471df-4308-4a65-8634-d177f94d0d4b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:30.392208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:30.392838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:30.393116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:30.393389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:30.405135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e1e59d59-eeaa-4ef0-9deb-692d505bff92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:30.410176 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:30.411166 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:30.412012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 242/479] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:33:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:30.417785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:30.550110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520f5ce-f275-429e-841c-f180c495ca2e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:30.591067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:30.592058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:30.592637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:30.593110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:30.593975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:30.594475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:30.596116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed06a76-7834-48c4-bd85-edafa199c568 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:30.606086 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 238/480] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:31.984041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095fe89-7294-42d5-b4ee-18da5bf2dc75 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:31.984534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095fe89-7294-42d5-b4ee-18da5bf2dc75 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:31.984534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095fe89-7294-42d5-b4ee-18da5bf2dc75 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:31.986171 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 243/481] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:31.993332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e538c0-3ba4-4730-9f63-e2362fbd1b45 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:31.993332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e538c0-3ba4-4730-9f63-e2362fbd1b45 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:31.993332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e538c0-3ba4-4730-9f63-e2362fbd1b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:32.020022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e8e538c0-3ba4-4730-9f63-e2362fbd1b45 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:32.021318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e8e538c0-3ba4-4730-9f63-e2362fbd1b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:32.044022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:32.124094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 239/482] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:32.124593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186dbb4-921a-4aae-b77e-bc3e19f147f2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:32.124877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186dbb4-921a-4aae-b77e-bc3e19f147f2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:32.125153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186dbb4-921a-4aae-b77e-bc3e19f147f2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:32.126753 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 244/483] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:32.132544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:32.132826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:32.133096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:32.163470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:32.164664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:32.258488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 240/484] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:32.603699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186dbb4-921a-4aae-b77e-bc3e19f147f2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:32.726786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:32.731832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186dbb4-921a-4aae-b77e-bc3e19f147f2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:32.767422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:32.768059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:32.768335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:32.768601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:32.775896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:32.787346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:32.793398 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:32.794271 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:32.795079 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:32.797490 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 245/485] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:32.804212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:32.909321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:32.935001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bab258-988e-47dd-98e1-1eb03b263ec4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:32.996869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:32.998904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.000668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.001150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.007657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2974a-916a-42b3-8aef-45524ba1c9ce None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.026446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-90529597-12bd-4537-9f48-dac9f8b3ae0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:33.031552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:33.032410 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:33.033203 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:33.035449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 241/486] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:33.036643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a405f-e8df-483f-98f4-34fa8190cbb7 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.037600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a405f-e8df-483f-98f4-34fa8190cbb7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.038084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a405f-e8df-483f-98f4-34fa8190cbb7 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.039506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 242/487] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:33.044216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a405f-e8df-483f-98f4-34fa8190cbb7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.066788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.068688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.069416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.070184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.071172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:33.071620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.073217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5125893-0fc9-4783-b9df-58ef6c40aaf7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.082978 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 246/488] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:33.085455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.085901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.086322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.106704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:33.108395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:33.125845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.169801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a405f-e8df-483f-98f4-34fa8190cbb7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.186979 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 247/489] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:33.188050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.235230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.235952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.236507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.236624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.237318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:33.237623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.239089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.257795 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 243/490] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:33.262955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.265931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.312683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdbbbf-a27f-45d1-a221-7622b43f0481 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.347993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.348632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.348908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.349209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.359579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.361027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-980f8d5f-aa22-49a5-9b45-dade9a8bf2a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:33.377555 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:33.378462 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:33.379451 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:33.379451 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:33.379451 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:33.379451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 248/491] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:33.382517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c8ab-4d12-49a0-bb93-ee2023f7a4f1 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.382517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c8ab-4d12-49a0-bb93-ee2023f7a4f1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.382517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66c8ab-4d12-49a0-bb93-ee2023f7a4f1 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.383085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 249/492] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:33.385016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66c8ab-4d12-49a0-bb93-ee2023f7a4f1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.388437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.404091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e2b5a-e4e3-4a2f-a581-f1524e88d6b0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.444103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.445219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.445711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.446270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.459301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.461531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1ebab9fa-11e8-4e2d-b284-47ea849c67e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:33.477751 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:33.478725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:33.479558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:33.479558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:33.479558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:33.481649 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 244/493] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:33.484729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f86e-becd-40de-bc13-845148073a02 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.485302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f86e-becd-40de-bc13-845148073a02 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.485765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f86e-becd-40de-bc13-845148073a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.506716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66c8ab-4d12-49a0-bb93-ee2023f7a4f1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.508172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c879f86e-becd-40de-bc13-845148073a02 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:33.509754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c879f86e-becd-40de-bc13-845148073a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:33.543156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.543775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.544055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.544336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.544964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:33.545266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.546744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.563538 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 250/494] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:33.564421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.575720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.596886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 245/495] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:33.598028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f86e-becd-40de-bc13-845148073a02 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.627116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.677075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4ea9c-0153-4b7d-b5a2-253de373bbe3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.714021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.714582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.714867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.715148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.715783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:33.716086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.717448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.734663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 251/496] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:33.735738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.740043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f86e-becd-40de-bc13-845148073a02 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.786370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.787733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.788154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.788763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.797807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.799304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:33.816481 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:33.817407 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:33.818954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:33.822582 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 246/497] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:33.823616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.862359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670c464-8768-482f-9e4e-b95b42a76d0b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.884523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.891509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:33.904380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.905010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.905285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.905552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.915885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:33.920634 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:33.921523 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:33.922302 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 252/498] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:33.922999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:33.949272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a10c2-646d-417c-953a-7cfdf3f17e18 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:33.982195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:33.982787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:33.983059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:33.983333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:33.991145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:33.992600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-77bb6803-691a-4fec-9589-5f91816beac2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:34.003634 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:34.004963 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:34.006521 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 247/499] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:34.007511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809698ec-d1fb-48c1-93be-662bd0726277 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.007511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809698ec-d1fb-48c1-93be-662bd0726277 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.007628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809698ec-d1fb-48c1-93be-662bd0726277 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.009059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 248/500] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:34.009887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809698ec-d1fb-48c1-93be-662bd0726277 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.030026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d5a8-abc9-49d4-9984-e71412ca3f8f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.063958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.064541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.064813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.065080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.065784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:34.066080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.067547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.105421 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 253/501] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:34.106804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.109972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809698ec-d1fb-48c1-93be-662bd0726277 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.112154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.145466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.146087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.146365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.146673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.147316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:34.147613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.148225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.149311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.157710 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 249/502] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:34.158485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.210146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8925e2-8d22-4f56-aa1e-3796c5eb8e77 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.243635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.244184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.244462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.244732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.255461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f614a9-f838-4ae2-acd1-13e61b9382f8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.255622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-bd8e2375-a717-4571-a87b-b1511b1bc418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:34.260361 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:34.261385 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:34.262181 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 254/503] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:34.263007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.263311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.263479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.280916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:34.282118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:34.295411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.296009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.296286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.296556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.297274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:34.297570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.299041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.339557 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 250/504] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:34.340374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.351276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 255/505] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:34.351276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.365704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.392194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.450837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de947-995c-40d1-9bf1-553879b255ee None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.470000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226ac66-5ddd-4753-a8f6-00f79a38a30b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.486758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.487362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.487641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.487917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.507429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.507934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.508180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.508446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.518605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.520051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:34.537206 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:34.538445 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:34.539471 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:34.539471 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:34.539471 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:34.539471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 256/506] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:34.544162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.565133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.566754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.592252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.627833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.650015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1086-045f-4e06-89e6-6075ee2e5a3c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.683576 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 251/507] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:34.684555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.689222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.689819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.690094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.690359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.691038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:34.691335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.692700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.701241 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 257/508] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:34.702015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.784055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3a29d-4129-4f61-83ac-e77ec045d255 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.804504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de80e1-3559-4059-a839-b6314d9afd3c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.807379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.818952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.819545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.819818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.820085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.839389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.839984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.840263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.840531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.841164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:34.841467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.842859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.854463 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 258/509] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:34.855261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.858709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:34.888015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:34.889489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:34.953013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96831b05-28e7-4013-aeea-d1afb5fcfe3b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:34.986922 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 252/510] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:34.987235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:34.987635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:34.988197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:34.988560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:34.988899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:34.999593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.001098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.013983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:35.015577 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.018803 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:35.019833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:35.019833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.019833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:35.019833 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 259/511] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.019833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.073427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.092262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782b6b3-0bcb-4561-805d-2ae77ffb254d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.117508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca948-2011-426b-884b-93cb16a66eb5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.126668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.127345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.127625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.127892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.138265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.139771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.150865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.151484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.151766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.152041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:35.154343 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.155261 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:35.156106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:35.156106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.156106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:35.156106 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 253/512] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.156314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.165192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.166686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:35.180784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.181681 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:35.183116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:35.183116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.183116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:35.183116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 260/513] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.183116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.236278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.260834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39221ba4-a674-45ad-b6eb-6864c2921693 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.282763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5befcb-4872-42af-8c12-1fb5ce2c303b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.291991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.294969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.295594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.295872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.296145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.303688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.305153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:35.316151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.317016 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:35.318468 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:35.319016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.319109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 254/514] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.319177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.319250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.319317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.319317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.332147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.333686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:35.348786 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.349705 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:35.355310 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:35.355310 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:35.355310 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:35.355310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 261/515] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.355310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.419286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092df0a-863c-4ef1-b7d8-0bf119146544 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.450303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09e2f2-d3b8-404f-b0f3-e91409acace6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.452370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.452967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.453238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.453501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.454127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:35.454422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.455829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.459976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.473425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 255/516] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:35.474860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.485149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.485747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.486025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.486300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.486962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:35.487264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.488622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.500783 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 262/517] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:35.501571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.525944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.576470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9627a0f-8859-4364-be3a-8eeeb6a1f98d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.606015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfdb49-5cf7-4f7a-92e8-c8d51e613d78 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.611138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.611737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.612013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.612282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.612909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:35.613210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.614606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.626485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 256/518] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:35.627288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.643278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.643909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.644184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.644858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.645576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:35.645952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.647791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.664621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 263/519] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:35.665394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.691316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.735317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24782d70-644e-4375-99bc-22604a552c31 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.743270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.765044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ced94-ac62-4617-8a74-01e78b7f07f5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.768588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.769192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.769472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.769886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.770550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:35.770898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.773309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.784756 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 257/520] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:35.785551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.803213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.803818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.804098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.804369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.804998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.806416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.827523 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 264/521] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:35 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:35.828331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.903520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea456a5-63da-4785-a0b7-ad222326940c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.922360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:35.941361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c42ca8-6e00-4dc1-b64f-a18f351fae52 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:35.943822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.944469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.944748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.945020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.952610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:35.954091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:35.966743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:35.967753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 258/522] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:35.968691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:35.987343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:35.987990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:35.988268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:35.988542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:35.991273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.000723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.002258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:36.019328 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:36.020494 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:36.021594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:36.022235 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 265/523] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:36.023808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.091932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b06e-5e77-4f7f-90d5-4820e02985ad None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.127846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.128451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.128728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.128999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.149288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dfd94-ce57-40d0-bf38-63ced808131d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.174969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.187080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.187717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.188000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.188274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.195125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.196575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.199388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.200853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:36.210343 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:36.211251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:36.212114 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 266/524] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:36.212592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.280248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.321506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 259/525] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:36.322328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.331612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf167203-6e51-4d2e-9bbb-466be5da54bd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.373024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.373805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.374188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.374596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.375388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:36.375795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.377485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.397307 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 267/526] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:36.398436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.403752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.428721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10ba1-f606-4284-a03d-71b84f3cf526 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.465344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.465938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.466218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.466486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.467228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:36.467526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.468988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.488082 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 260/527] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:36.489016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.509294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a29a41-22d0-4c8f-bea6-3c9e6d7f2ae4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.524794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.547504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.548156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.548489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.548783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.549449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:36.549754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.551233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.568924 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 268/528] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:36.568924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.594809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b86181-f1fb-4e6a-8ca0-d9b014292a70 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.633113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.634352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.635095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.635552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.638763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.650037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.651644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:36.667018 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:36.668128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:36.668964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:36.668964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:36.668964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:36.668964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 261/529] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:36.668964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.686916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8bb23-76b7-43f7-80d0-4ab8f4d4e0ae None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.721864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.722467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.722776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.723050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.723766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=99f0d8a1aa0d4bacbd4e4fb2e7864cee) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:36.724617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.726177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.731520 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 269/530] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/role_assignments?user.id=99f0d8a1aa0d4bacbd4e4fb2e7864cee&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:36.732296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.776333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.809328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7d3ac-cc2d-4fd1-ad87-7ba5931ca606 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.848463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9c900-93ea-4f6c-9543-0a04f2333505 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:36.859354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.860385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.861383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.862147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.888025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:36.888636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:36.888916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:36.889189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:36.889921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:36.890226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.891745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:36.903025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 270/531] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:36.904031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:36.944985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:36.963064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:36.964965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.134911 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 262/532] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:37.140632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.143016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9525b-5fd6-4ead-a330-7592b451f412 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.145411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:37.193902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.195400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.196181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.196663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.197506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:37.198028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.200093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.213846 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 271/533] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:37.219256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.321835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fc1a-d1aa-438a-a022-b5e7a38876be None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.361892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.362988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.363499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.363981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.376337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.379031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.386839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:37.402928 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:37.403874 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:37.404730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 272/534] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:37.412538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.419483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8a759-8f8c-4957-a849-ce09fa1862eb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.464727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:37.487031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade467-d19f-455b-abea-b333260048b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.487677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade467-d19f-455b-abea-b333260048b2 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.487951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade467-d19f-455b-abea-b333260048b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.488220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feade467-d19f-455b-abea-b333260048b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.509433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade467-d19f-455b-abea-b333260048b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.511237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feade467-d19f-455b-abea-b333260048b2 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-feade467-d19f-455b-abea-b333260048b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:37.542715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:37.543690 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:37.544531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:37.544531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:37.544531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:37.546673 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 263/535] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:37.552497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade467-d19f-455b-abea-b333260048b2 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.597522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc305c-7669-4678-be3a-2be42c24b41a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.664770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.665402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.665674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.665953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.666689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:37.666999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.668450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.682922 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 273/536] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:37.691037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.736730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feade467-d19f-455b-abea-b333260048b2 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.797538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.798153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.798432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.798730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.799373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:37.799676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.801072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.814848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:37.815016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:37.827825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 264/537] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:37.834668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:37.848888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac69b2b-15c5-4e9f-a516-da88919caf2a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:37.893054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:37.893660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:37.893940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:37.894210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:37.894955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=c66ad10d7496429da3b81648cf9e83f3) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:37.895262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:37.897056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:37.903377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 274/538] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/role_assignments?user.id=c66ad10d7496429da3b81648cf9e83f3&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:38.013563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a9546-9299-44de-b711-b1c7b6ea4d8b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:38.073186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:38.073831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:38.074109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:38.074376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:38.089444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:38.092292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:38.113207 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:38.114111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:38.115039 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 265/539] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:38.121905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:38.233191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a02f32-097a-4a3c-acef-e8dcb3038a1b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:38.267981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:38.268573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:38.268856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:38.269129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:38.269814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:38.270121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:38.271778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994077b-6b6b-407a-931d-5c88ce06b2a9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:38.285025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 275/540] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:38.295612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:38.378769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:38.438559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27af61-dded-4122-b959-bc8a56c64ab2 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:38.488087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:38.488696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:38.488977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:38.489248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:38.490588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b98deaef6c24ed2aa7c7ecc90c75205) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:38.490897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:38.492391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb772d2a-4510-4af9-95a7-181ed9dc582b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:38.499494 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 266/541] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:38 2022] GET /identity/v3/role_assignments?user.id=0b98deaef6c24ed2aa7c7ecc90c75205&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:38.589181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba135-4773-4a64-b9e2-2fe1e17eacf8 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:38.589497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba135-4773-4a64-b9e2-2fe1e17eacf8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:38.589745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba135-4773-4a64-b9e2-2fe1e17eacf8 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:38.591120 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 276/542] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:38.596685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:38.596975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:38.597251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:38.622531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:38.623752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:38.741328 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 267/543] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:39.117278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ba135-4773-4a64-b9e2-2fe1e17eacf8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:39.126790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:39.232925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ba135-4773-4a64-b9e2-2fe1e17eacf8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:39.270416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:39.272558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.273305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.273791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.292954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:39.298146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:39.299030 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:39.299820 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:39.302129 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 277/544] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:39.308138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:39.366400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:39.431282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9162f-db5c-4a08-91fe-fec150d30874 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:39.481814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:39.482892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.483403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.483885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.484767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:39.485267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:39.486894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:39.497284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 268/545] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:39.503011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:39.595097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:39.631507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027d647-d350-4200-8be9-b9585187e6d5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:39.686163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:39.688195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.688485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.688755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.706214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:39.707783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e5396123-7f09-4702-9b95-83efc36bab54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:39.723505 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:39.724959 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:39.726231 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:39.726231 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:39.726231 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:39.726231 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 278/546] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:39.732229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:39.781730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b1fe-c0a2-4529-9012-7662b25f7698 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.782042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b1fe-c0a2-4529-9012-7662b25f7698 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.782308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b1fe-c0a2-4529-9012-7662b25f7698 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.783884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 279/547] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:33:39.790146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.790431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.790736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.810235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:39.811475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:39.889253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03c29-311d-4e02-8d87-b86e619d47a1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:39.901765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 280/548] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:39.923656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:39.940224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:39.940808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:39.941090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:39.941362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:39.942008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:39.942309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:39.943935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb63a9-cb43-4e18-950a-335ac9845a68 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:39.956444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 269/549] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:39.962794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.028675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.078881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12080a-7c1f-4f10-9f17-692a402405a0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.079250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12080a-7c1f-4f10-9f17-692a402405a0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.079430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12080a-7c1f-4f10-9f17-692a402405a0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.081179 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 270/550] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:40.094467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc82e-0146-454a-acfb-5f093d91dcab None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.098124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.098802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.099795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.131418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.132400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.132905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.133405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.141102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:40.142823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:40.154521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:40.154521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:40.164913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:40.165800 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:40.166726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:40.173476 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 281/551] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:40.174945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.283697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab242dad-9d1e-4452-967e-e01ebd4214c4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.294700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.295592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 271/552] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:40.321239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.322181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.322710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.323204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.324123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:40.324627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:40.326249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:40.349468 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 282/553] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:40.351385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.387473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.527817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03668022-2db0-4787-9c9f-bbb08a35d9cc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.531843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0231-562a-4890-a1de-9ba5ae44d6b2 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.535251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.589293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.589908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.590188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.590463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.592144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.592769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.593047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.593318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.623619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:40.628584 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:40.629548 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:40.630659 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:40.640227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 283/554] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:40.646690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.678463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.715606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:40.717553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:40.765344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a04847-779a-4be8-8f61-65f00a0d6b6a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.804717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.805249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.805535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.805807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.806479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:40.806813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:40.808179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:40.822282 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 284/555] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:40.822282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.840245 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 272/556] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:40.841432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:40.846081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.919069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:40.934809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3ecc-82b8-4dff-9b20-0a43c9da424c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.946988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7834ad3-cce0-4b16-8476-b49e61a6073c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:40.972156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.972774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.973108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.973380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.986973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:40.987188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:40.988090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:40.988573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:40.989093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:40.992971 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:40.993948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:40.995309 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 285/557] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.000025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.007105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.008958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:41.025740 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:41.026836 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:41.027731 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:41.027731 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.027731 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:41.034202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 273/558] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.035421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.095267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.109124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21adda-3704-43f7-b17c-8fdea4ee5b79 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.139695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71987e-aa36-457a-a272-6e827f59f0f9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.143386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.144012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.144291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.144561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.155775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.157245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.165982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:41.173745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:41.174804 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:41.175656 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:41.175656 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.175656 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:41.175656 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 286/559] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.175656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.181248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.181869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.182149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.182422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.183129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:41.183435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.184816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c675454-a08a-47b1-88bf-8cf72a2cc50e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.198068 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 274/560] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:41.198068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43fdb-fd50-4dd8-8a1b-fba4767cd276 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.198068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43fdb-fd50-4dd8-8a1b-fba4767cd276 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.198068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e43fdb-fd50-4dd8-8a1b-fba4767cd276 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.198068 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 275/561] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:41.198470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e43fdb-fd50-4dd8-8a1b-fba4767cd276 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.289824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19841a56-dc5b-4842-a674-4d5389b593ce None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.303508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e43fdb-fd50-4dd8-8a1b-fba4767cd276 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.321005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.326339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.326994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.327277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.327551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.328195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:41.328493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.329875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.338941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.339537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.339818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.340094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.340731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:41.341034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.341977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 287/562] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:41.344636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.344751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939b4c-977a-4590-8c57-3b246c87e635 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.356229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 276/563] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:41.358246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.358791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.359262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.377946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:41.379185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:41.420148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.455176 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 277/564] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:41.456100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.463709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081ec4-5009-4ea8-80a2-91a0c7ad2106 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.504804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.505471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.505753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.506047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.517202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.518718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:41.534784 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:41.535894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:41.537958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:41.537958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:41.537958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:41.537958 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 288/565] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.537958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.566165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb684bd-c85c-4e0a-b4a0-126a1eb8524a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.576205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.603963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.604872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.605156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.605416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] PATH_INFO: `/v3/users/glance` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.612624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.614016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-019c363a-e5de-4e1e-b208-8540c067f79c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:41.625578 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:41.626526 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:41.627444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 278/566] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.628044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678566-ad3b-4387-ac07-aff69c15ab0f None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.628044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678566-ad3b-4387-ac07-aff69c15ab0f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.628497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83678566-ad3b-4387-ac07-aff69c15ab0f None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.637850 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 279/567] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:41.638464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83678566-ad3b-4387-ac07-aff69c15ab0f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.658343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920fe4e-5c53-4715-84c1-e06b3e4821a8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.671331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.693344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.694340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.694652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.694948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.701888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.703333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:41.714097 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:41.715187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:41.716041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 289/568] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:41.716510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.757349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83678566-ad3b-4387-ac07-aff69c15ab0f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.793521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.794132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.794400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.794700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.795335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:41.795639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.797023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37637943-f8e1-42b7-825c-c572934bb2a0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.808536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 280/569] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:41.809710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.809961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.810233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.820056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849153-f633-4e6d-b7a3-3451d9705305 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:41.827822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:41.829003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:41.851477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.852829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:41.853420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:41.853697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:41.853965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:41.854703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:41.855007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:41.856454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:41.873511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 290/570] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:41.874245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.901109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 281/571] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:41.901880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:41.909377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:41.982769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dd1d2-dd74-4d59-82e9-f19c25683bec None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.016738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a41f89-56ab-4990-af1e-7d7ef8d571bd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.019379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.019984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.020256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.021048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.021757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.022059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.023747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.040511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 291/572] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.040511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.056498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.057476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.057813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.058182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.070182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:42.075171 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.076106 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:42.077010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 282/573] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.078317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.100305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.143442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.154880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5729b3-eeea-4e93-b0b2-5f308c584532 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.190058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.190706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.190990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.191260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.199504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.200921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.201097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fefc4-1659-479a-a159-03ca45ed9719 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:42.212919 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.213872 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:42.214826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:42.218771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 292/574] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.219402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.240046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.240827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.241212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.241573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.251463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.252949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:42.267263 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.268371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:42.269237 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:42.269237 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.269237 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:42.269237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 283/575] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.269237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.324435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955fc66-5e8c-4d52-a5fd-532c057cab9f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.337841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.357389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.357987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.358260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.358549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.359300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b5a7880d6ec47f381af9ef1f09a767c) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.359577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.360994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.367041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 293/576] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/role_assignments?user.id=0b5a7880d6ec47f381af9ef1f09a767c&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.367805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.370597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b11e7-05fc-4075-ba45-978a33b30343 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.400230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.410979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.412262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.412606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.412961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.413717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.414087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.415677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.429340 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 284/577] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.429340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.501569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988f98b-e8c3-40d5-9e60-0d8b2928fffc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.540116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad1c2-36a2-496a-ac77-24e144595624 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.541565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.542258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.542620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.542955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.543756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.544129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.545678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.566439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 294/578] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.571947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.573441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.580661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.581297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.581571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.581838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.593321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:42.598976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.600284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 285/579] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.601496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.658912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.694140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4061c-bbbc-4183-9bf3-ed60f541ecfd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.706961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9377240-4a5f-45c7-b185-4b6196f9076a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.732006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.732621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.732900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.733173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.733803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.734105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.735509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.741277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.742015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.742403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.742796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.751980 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 295/580] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.751980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.757107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.758670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:42.773258 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.774530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:42.775627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:42.775627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.775627 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:42.780880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 286/581] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.780880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.827984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.865601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29acb8-7792-43b6-adc0-1baf8c16e043 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.887243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72957021-5117-4f8a-845e-a766a2d8b294 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:42.905641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:42.911528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.912377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.912490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.912751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.923273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:42.923873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:42.924146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:42.924413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:42.925067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:42.925185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.925364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:42.926810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.926921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:42.934975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 287/582] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:42.935778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:42.946938 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:42.948071 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:42.954063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:42.954063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:42.954063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:42.954063 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 296/583] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:42.954063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.043804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17197954-c31f-42e3-a5a4-e023ab040230 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.050883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.060700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c035-ed08-4760-8f2e-9e53df518851 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.083973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.084680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.084955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.085224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.092026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.093764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.096852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.097632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.098052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.098667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.099510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.099998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.101503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:43.104952 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:43.105969 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:43.107078 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:43.110596 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 288/584] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:43.111381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.118045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 297/585] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:43.119118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.168632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.225044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fc0d9-ca9c-4540-bb47-6c2133fc82cb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.226626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f82ee-4fab-48cf-adb4-45f23dff387f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.258719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aec57d-29ac-404d-848f-181311face95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.259343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec57d-29ac-404d-848f-181311face95 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.259620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec57d-29ac-404d-848f-181311face95 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.259891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aec57d-29ac-404d-848f-181311face95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.262423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.263050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.263326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.263597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.264230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.264526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.265905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.270876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aec57d-29ac-404d-848f-181311face95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.272317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aec57d-29ac-404d-848f-181311face95 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.282246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 298/586] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:43.282246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-74aec57d-29ac-404d-848f-181311face95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:43.291092 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:43.292013 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:43.292873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:43.292873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:43.292873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:43.292873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 289/587] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:43.293733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aec57d-29ac-404d-848f-181311face95 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.312506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.390541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d34590-3df8-4ab4-a115-c087942936ce None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.393469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.396334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aec57d-29ac-404d-848f-181311face95 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.428304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.428837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.429114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.429384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.430072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.430374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.431862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.432177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.432926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.433304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.433696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.442942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.444703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.451655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 299/588] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:43.451655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:43.461473 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:43.462641 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:43.463804 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 290/589] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:43.468696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.555575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.564869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4d4b4-34ac-467d-b1c2-30a887385d9a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.582392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22f25f-6363-4711-ab63-7ca6ce7b65f5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.601322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.602290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.602767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.603190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.610674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.612215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.616000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.616661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.616942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.617212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.617847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.618155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.619557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:43.624550 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:43.625406 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:43.626266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:43.631303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 300/590] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:43.632793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.633998 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 291/591] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:43.635073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.666153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.740772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79c746-6ebe-4e01-adb9-e67715989521 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.749692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00848a9c-7c99-4f96-8ceb-5e86f6d4ea27 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.779356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.780016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.780298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.780574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.781293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.781609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.783080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.786952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.787612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.787891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.788163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/99f0d8a1aa0d4bacbd4e4fb2e7864cee/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.793881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.803752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=99f0d8a1aa0d4bacbd4e4fb2e7864cee, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.805203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:43.810462 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 292/592] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:43.810462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.821074 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 301/593] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:33:43 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/99f0d8a1aa0d4bacbd4e4fb2e7864cee/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:43.830677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:43.926802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3b40d-8d6a-4a5e-b0f9-90cc6e20260f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.943196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:43.971367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:43.972143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:43.972510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:43.972872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:43.974190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:43.974510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d77e9c-932b-481b-8a92-fe8536a7ea33 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:43.974877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:43.976563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.002761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 293/594] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:44.003597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.024158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.024799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.025076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.025346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.035610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.038739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:44.056965 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:44.058124 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 302/595] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:44.059391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.087596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.148355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb2dd-db2a-4f10-b6fb-74105e9431f5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.189629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df072e8d-4137-437c-a86f-85ecaf8b6d16 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.195660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.196575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.197048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.197553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.213304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.215301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.228063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.228817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.229165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.229561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.232339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:44.237259 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:44.238558 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:44.240157 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:44.240157 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:44.240157 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:44.242703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 294/596] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:44.243680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.253765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=6531dafbedef45ccb7b6b7e2f610c2c2, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.256194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.273864 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 303/597] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:33:44 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:44.280355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.426379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220294e-6b1e-457f-bc84-6a4facf60ca8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.443893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.474911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.475542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.476234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.476529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.477873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:44.478339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.480656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.503223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3170c3c-a837-4014-82ce-0a38575c766a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.520959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 295/598] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:44.528096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.576791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.577395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.577671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.577939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.579352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:44.579352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.580414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.596894 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 304/599] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:44.602370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.663360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.681586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40956484-026a-4880-af55-9f1e50b0484b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.726132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988403a-d109-4e24-ac41-ba112a6d3f26 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.731109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.732300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.732777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.733242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.748222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.749997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:44.769295 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:44.770749 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:44.772087 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:44.772087 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:44.772087 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:44.774918 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 296/600] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:44.783675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.822820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.823012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.823530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.824124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.824124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.837129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.838604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-7a292ae6-ecc5-40f1-8949-ee995f0103d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:44.852216 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:44.853296 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:44.854305 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:44.856021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 305/601] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:44.857198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e97-a24c-4cb5-92a3-f769017401dc None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.857479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e97-a24c-4cb5-92a3-f769017401dc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.857748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e97-a24c-4cb5-92a3-f769017401dc None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.861320 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 306/602] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:44.861320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8e97-a24c-4cb5-92a3-f769017401dc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:44.947423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df27cdf-3b76-4ab4-8aa6-2c139393336f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.986692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8e97-a24c-4cb5-92a3-f769017401dc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:44.990328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:44.992987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:44.993598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:44.993871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:44.994143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:44.994809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:44.996008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:44.996989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f3e67-6dde-412b-b6dd-b977ecf2fb9d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:45.018355 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 297/603] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:45.018355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.018355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.018355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.034225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.035085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.035524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.035967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.037158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:45.037605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:45.038819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:45.039382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:45.039990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:45.055101 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 307/604] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:45.056231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.128263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 298/605] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:45.129381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.151320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:45.342062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3fb4f-a283-42c6-a141-4d30c9508dd9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.393193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.393685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e682c-672d-4831-964f-97d4e3fe3663 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.393864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.394127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.394400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/c66ad10d7496429da3b81648cf9e83f3/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.412979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=c66ad10d7496429da3b81648cf9e83f3, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:45.414896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:45.442874 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 299/606] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:33:45 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/c66ad10d7496429da3b81648cf9e83f3/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:45.450579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.451304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.451585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.451856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.460939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:45.462531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:45.476217 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:45.477151 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:45.478020 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 308/607] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:45.483620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.533651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.578910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:45.608508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:45.697662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a4934-7f92-4ea9-8de8-739bc65ea067 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.718946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ca1ab-9a0b-4d61-ae8d-b56149ee5b6c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.756891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111eace-f443-489f-bcda-3094379a868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.757562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111eace-f443-489f-bcda-3094379a868c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.758261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111eace-f443-489f-bcda-3094379a868c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.759118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111eace-f443-489f-bcda-3094379a868c None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.759987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111eace-f443-489f-bcda-3094379a868c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:45.760304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111eace-f443-489f-bcda-3094379a868c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:45.761748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111eace-f443-489f-bcda-3094379a868c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:45.763753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.764501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:45.764781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:45.765048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:45.776749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 300/608] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:45.781450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:45.782156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111eace-f443-489f-bcda-3094379a868c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:45.787383 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:45.788386 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:45.789186 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 309/609] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:45.794614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:45.941982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:45.952747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111eace-f443-489f-bcda-3094379a868c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.961756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5408e7-c061-4110-a2a7-41b98c17a627 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:45.999438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:45.999683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:46.000515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.001022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.001504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.002825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:46.003325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.004949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.014574 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 310/610] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:46.020555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:46.047203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:46.047883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.048416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.048416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b98deaef6c24ed2aa7c7ecc90c75205/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.075470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=0b98deaef6c24ed2aa7c7ecc90c75205, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.076953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.104978 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 301/611] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:33:45 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b98deaef6c24ed2aa7c7ecc90c75205/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:33:46.134152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eef3f-a336-4989-ad5c-3534e4b0109f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:46.178303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:46.179284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.179743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.180222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.199074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.200848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.216027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:46.231357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:46.232369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:46.233229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:46.233229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:46.233229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:46.235346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 311/612] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:46.241362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:46.382619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981aab5-b11e-4edf-b65d-163e89faf64e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:46.435218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:46.435934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.436210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.436486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.437123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:46.437418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.438804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.452481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 302/613] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:46.458365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:46.586907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb14d51-3b5a-43d4-b536-91569c4a150d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:46.622117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:46.622773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.623051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.623318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.644959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.647002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-08ede109-5fc2-4b6a-b8cc-abad2517fcf4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:46.662064 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:46.664258 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:46.666439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 312/614] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:46.671798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:46.721514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697cc9-64de-47f1-b48b-96b15be2def1 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.721814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697cc9-64de-47f1-b48b-96b15be2def1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.722010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35697cc9-64de-47f1-b48b-96b15be2def1 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.723134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:46.724085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 313/615] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:46.729366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.729667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.729949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.752014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:46.753335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:46.818948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:46.830520 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 314/616] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:46.854420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64296184-b0bc-412e-acf7-b64ab43f693e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:46.904012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:46.904612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:46.904886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:46.905152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:46.905861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:46.906160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:46.907616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:46.936219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 303/617] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:33:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:46.941603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.087778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a4e8d-4b34-453f-b61a-76847aeff723 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.134736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.135765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.136245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.136737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.151026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:47.152822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:47.178799 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:47.179780 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:47.180665 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:47.180665 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:47.180665 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:47.182736 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 315/618] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:47.191576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.247266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.322913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:47.342261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:47.359140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8ff43-329c-44a3-8b00-160a1809db6f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.360553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8529a-52e7-4167-b03a-2c3ef37632d5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.396815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.397681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.398133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.398417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.398865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.399121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.399399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.399670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.400206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:47.400653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:47.402434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceab5a3-921c-4ba0-bc3d-02c588bc1cb1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:47.412704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:47.415754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 304/619] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:47.417252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580677f0-02cc-4e30-bc04-3e5fa6206351 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.417707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580677f0-02cc-4e30-bc04-3e5fa6206351 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.418146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580677f0-02cc-4e30-bc04-3e5fa6206351 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.419593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 305/620] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:47.425068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.425556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.425996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:47.427347 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:47.428477 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:47.429459 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:47.433222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 316/621] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:47.434034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.450802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:47.452459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:47.529209 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 306/622] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:47.532418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.554653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:47.559243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab531d42-f0f7-4956-a918-d56f511a346f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.624401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.625769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.626960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.627576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.646545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:47.646545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:47.668042 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:47.669146 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:47.670240 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:47.670725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 317/623] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:47.678059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.684891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:47.725860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a901aa-32a9-4e13-ac1b-3cf4138b3266 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.792890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.793507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.793785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.795472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.796161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:47.796468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:47.797858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:47.816574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 307/624] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:47.822423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.851109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61272-c5d2-4602-b007-f80983e539fc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:47.915441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:47.915982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:47.916262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:47.916538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:47.917233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:47.917538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:47.919077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:47.944674 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 318/625] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:47.946138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:47.972626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:47.994210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ae9d8-07c1-4a1b-82e3-26b51385c8f3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.033067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.033703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.033977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.034244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.044618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.050927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.053906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.072398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaaa04-176c-4fb1-a93e-65f4bb6ad748 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:48.074879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:48.075812 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:48.076698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:48.076698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:48.076698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:48.080553 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 308/626] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:48.081373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.117604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.118882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.120252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.120735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.134150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:48.140785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:48.141684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:48.142541 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:48.150760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 319/627] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:48.150760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.203251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7f4e1-b56a-49b9-828c-c6417d90f9b7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.245305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.246325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.246779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.247067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.247703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b5a7880d6ec47f381af9ef1f09a767c/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.267209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=0b5a7880d6ec47f381af9ef1f09a767c, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.269419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.270883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e776a5-96d6-4195-92da-67319663dfbd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.286626 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 309/628] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:33:48 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b5a7880d6ec47f381af9ef1f09a767c/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:33:48.287817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.334830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.335608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.336040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.336402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.337149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:48.337546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.339084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.341024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.367432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 320/629] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:48.373400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.424193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f1b42-9f5a-4276-a43a-36b91c234708 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.477116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.477902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.478181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.478446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.479341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:48.479450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.480903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7f320-4524-4cb2-83d7-85e770275698 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.492478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 310/630] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:48.497441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66de20-2e6b-447e-80b4-57c12462c1e6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.497441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66de20-2e6b-447e-80b4-57c12462c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.497441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66de20-2e6b-447e-80b4-57c12462c1e6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.497441 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 311/631] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:48.498540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66de20-2e6b-447e-80b4-57c12462c1e6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.522166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a78aef-eba6-4a4b-b020-f28201262d83 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.561612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.561984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.562258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.562561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] PATH_INFO: `/v3/users/glance` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.571770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.573218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.576002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c9bef54d-8b6f-45ed-a6e7-7b7e0bbd3a88 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:48.587877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:48.588854 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:48.589840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 321/632] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:48.590431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.591072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.591072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.612044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:48.613222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:48.639597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66de20-2e6b-447e-80b4-57c12462c1e6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.642089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.688699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.690119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.690888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.691362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.707613 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 322/633] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:48.707613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.715431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.716764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:48.752230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:48.753215 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:48.754114 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:48.754114 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:48.754114 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:48.754114 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 312/634] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:48.760353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:48.901880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273aab2-52d4-483d-8c40-744b0adc1527 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.933273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ea63f-034b-4ad2-aa46-9b136d7ba1a2 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:48.960655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:48.969046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.969652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.969934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.970208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.970963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:48.971272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.972713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:48.982748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:48.983348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:48.983623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:48.983886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:48.984522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:48.984821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:48.986174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.014976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 323/635] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:49.015589 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 313/636] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:49.016207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.030431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.071671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.150006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc8b15-3730-4634-ada0-0fcc9a7d42de None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.197713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.197928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cb0bf-530f-4b69-adc8-f147fbb3788e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.198327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.198643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.198919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.235561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.236521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.236521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.236744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.244885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.246324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.258119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:49.258571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:49.264151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 314/637] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:49.265276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.280433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.281983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3898c-1ceb-4d78-bdf7-b60bddf444d9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.346881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.376579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b149a-3746-49b8-8d7c-d739b0888489 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 324/638] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6be87-2101-4bd0-a8b7-a2eff05ed1ab None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6be87-2101-4bd0-a8b7-a2eff05ed1ab None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6be87-2101-4bd0-a8b7-a2eff05ed1ab None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 325/639] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:49.391861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6be87-2101-4bd0-a8b7-a2eff05ed1ab None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.419242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.419881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.420158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.420422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.431101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:49.435913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:49.436842 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 315/640] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:49.438583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.497582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.511392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6be87-2101-4bd0-a8b7-a2eff05ed1ab None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.546821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.547696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.548136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.548648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.549517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:49.549978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.551855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c57b9-d546-4365-92c2-022e905be238 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.552283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292c1e2-c50c-40f6-a89f-4964fe505df5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.577300 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 326/641] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:49.577300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.577300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.577300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.593519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.593842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:49.594145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.594421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.594734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.595010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:49.596904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.607629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.609074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:49.625609 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:49.626725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:49.627657 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:49.627657 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:49.627657 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:49.627657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 316/642] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:49.627657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.686677 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 327/643] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:49.686677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.755238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb2064-ceba-4319-bac6-1f3445222784 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.774526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.802069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e747c90-c8de-47d9-a0bd-29e313987497 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:49.806554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.807407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.807882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.808351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.809208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:49.810267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.812347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.824240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 317/644] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:49.825400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:49.844535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:49.845147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:49.845425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:49.845695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:49.895882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:49.930406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:49.932370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:49.966245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19506a15-4690-4b43-9748-b572d14b9467 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.010476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.011117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.011392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.011660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.012294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:50.012590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.014354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.016263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.034660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 318/645] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:50.036170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.046684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 328/646] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:50.047792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.141285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e51d-31fe-4acb-99b3-e3b8af95ccdd None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.151304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.151550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f810b-4571-41df-ae3c-e47cc2f0ef76 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.180631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.181599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.181911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.182163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.189841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.190476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.190784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.191053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.194628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.197078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.202920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:50.208220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.209222 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:50.210104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:50.212015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 329/647] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.212927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:50.213213 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.214126 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:50.215034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:50.215034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.215034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:50.218690 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 319/648] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.219499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.259370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.327676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bf26c-47b0-4ea9-81a7-21e700d5691b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.334315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098712-683a-46f3-9ef6-95cf12ae0ce6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.362716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.363343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.364145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.364414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.370173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.370833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.371114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.371444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.375455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.376906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.378129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.380187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:50.391062 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.392065 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:50.393603 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:50.393603 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.393603 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:50.393603 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 320/649] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.393603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:50.397570 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.398409 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:50.399382 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 330/650] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.400558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.415103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.508903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.515876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fe1ab-5664-4ca2-84d4-bc4bf1efa9c8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.518927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1fee4-a3ed-42ae-bf80-6867efe0ccbb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.551580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.552212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.552488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.552761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.553459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:50.553796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.554440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.555072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.555167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.555351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.555623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.556277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:50.556576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.557954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.566744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 321/651] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:50.567899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.571887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 331/652] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:50.572643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.649028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.680885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac3691-3b75-4ff3-9dcb-70e85499692c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.688372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b43d3-4d01-442c-8ac6-7d611a3aa8be None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.717040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.717639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.718168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.718168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.718824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:50.719116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.720482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.730683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.731362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.731970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.732312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.737355 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 322/653] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:50.737509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:50.737509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.737669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.737773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.749182 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 332/654] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:50.750073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.769548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.848736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261c33-84c9-4c90-bf2a-ca57c684b5d7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.855982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c17cec-dffa-4740-b5bd-81f38655a8f6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:50.885039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.885665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.885943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.886214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.888283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:50.892580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:50.893214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:50.893501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:50.893773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:50.898652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.900147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.902037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:50.903488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:50.914838 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.915920 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:33:50.917138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:50.917672 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:50.918546 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:50.919523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:50.919523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:50.919523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:50.919523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 323/655] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.919523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.920588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 333/656] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:50.921228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:50.996530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.016880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3510ee-307e-4dc8-adb9-aff03283cbf9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.018903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0e96c-da0d-4766-9800-a5ec922224fb None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.052258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.052746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.052895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.053161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.053337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.053455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.053813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.054088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.054825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=99f0d8a1aa0d4bacbd4e4fb2e7864cee) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.055130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.056526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d9c76-b3aa-4455-ba39-f4279890f974 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.060227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.061642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.062313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 334/657] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/role_assignments?user.id=99f0d8a1aa0d4bacbd4e4fb2e7864cee&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.063250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387615be-0689-4808-a902-cee6ba5795de None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.063531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387615be-0689-4808-a902-cee6ba5795de None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.063796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387615be-0689-4808-a902-cee6ba5795de None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.064952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 335/658] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:51.065633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387615be-0689-4808-a902-cee6ba5795de None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:51.086095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:51.087309 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:51.088284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 324/659] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:51.089022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.139539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.198864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f8154-dccd-4c48-8286-a3cb6aff45f9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.201125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387615be-0689-4808-a902-cee6ba5795de None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.235819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.236531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.237163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.237544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.238904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.239573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.241189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8b6fc-9504-46b5-821f-84f2735ad69c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.242290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.242941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.243251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.243820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.244503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.245740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.247151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.257048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.266220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 325/660] 10.222.0.52 () {56 vars in 1137 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.266220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.266220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.266220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.266680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 336/661] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.268481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.287694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:51.289027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:51.363286 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 326/662] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:51.365125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.388143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38cb9-0298-443b-b827-b534397056c9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.419542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.430271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.431319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.431807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.432499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.433524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.434066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.435725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.450660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 337/663] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.451533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.512515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1942ec6-b5a1-4d03-b83b-ad685abce6e1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.534447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.550013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.551612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.552101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.552574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.560606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.562695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:51.575049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:51.575994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 20:33:51.576929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:51.579434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 327/664] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:51.580477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.584267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5645624-86a7-427f-9c70-56f306eecbc0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.630040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.630758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.631038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.631469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.687587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.696356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b93e9e-c402-4e03-9220-a69f4395ddb3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.716615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.717980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.735743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.736361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.736618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.736885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.737597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.737898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.740079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.746297 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 328/665] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.747125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.795852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:51.856389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 338/666] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:51.858681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.898713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36035535-8cd5-45fa-9ac5-5bf659359038 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:51.945874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:51.946716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:51.947090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:51.947459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:51.948362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:51.948761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:51.950782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:51.976801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 329/667] 10.222.0.52 () {56 vars in 1139 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:51.977860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:51.991592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0af230-2591-454d-b737-53df9f39aec0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.001287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.034475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.035613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.036858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.037353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.054228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:52.059833 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:52.060745 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:52.061574 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 339/668] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:52.062278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.093096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.106364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722399f-3d57-4959-8391-82177a28e82f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.145995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.146679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.147686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.147987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.161210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.163308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.183680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57e27b-787d-4f08-8067-57c44254c56b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:52.184674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:52.185555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:52.186421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:52.186421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.186421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:52.190521 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 330/669] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:52.190521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.224713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.225363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.225657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.225904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.284733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.306165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.307810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.329375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.330199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4c977-130a-42b5-890a-5fda00868379 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.368187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.369088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.369508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.370157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.371093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:52.371535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.373165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.386783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 331/670] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:52.387999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.414996 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 340/671] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:52.415949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.505149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a57b48-eb7e-498d-b951-3c3131f9c761 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.524869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957d684-f082-42d5-8406-5637f12fb65d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.538934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.546136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.546891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.547523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.547785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.548049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.548871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:52.549177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.551170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.566972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.567863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.568209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.568477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.569186 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 332/672] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:52.570268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.579738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.581537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:52.599210 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:52.600225 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:52.601142 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:52.601142 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.601142 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:52.602894 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 341/673] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:52.603929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.701408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a7fe8-33ea-45ef-ab91-afddc18f05f0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.740044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fb31f-aba3-4432-95e8-4afa42263bb5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.749434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.750275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.751136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.751555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.766745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.768885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.778205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.779486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.779908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.780276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.788251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:52.789239 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:52.790318 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:52.791429 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:52.791429 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:52.791429 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:52.792049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.793692 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 333/674] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:52.795506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:52.805226 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:52.806283 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:52.807392 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:52.809153 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 342/675] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:52.810106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.846082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.852240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:52.924334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231f45b-efb9-4de6-910a-293984170deb None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.932901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0582-d111-4c5f-a2a4-3b1fb08ce266 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:52.964269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.964876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.965157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.965428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.966079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:52.966365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.967805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:52.978047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:52.978981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:52.979736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:52.980204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:52.981035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:52.985775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:52.985908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 334/676] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:52.985908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:52.987846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.002654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 343/677] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:53.003824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.096645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b55203-008a-4a01-9f2c-76610015dcaf None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.098753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.116963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.130149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f858815-9061-4d14-98a2-9aa7d71af14e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.134839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.135448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.135726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.135997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.136685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:53.136989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.138440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.156312 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 335/678] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:53.157215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.171732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.172764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.173453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.173930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.187588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.189780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:53.208843 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:53.209877 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:33:53.210806 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:53.213978 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 344/679] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:53.215304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.273706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25bc7-3d93-41d6-b2c0-3188c3736e1f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.309492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.310125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.310402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.310705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.318732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.320168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:53.331550 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:53.332571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:53.333551 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:53.337676 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 336/680] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:53.338555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.349174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2c77b-6cb1-4ecc-a992-cc7bf3afdf4c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.364531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.395771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.397299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.397778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.398334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.399365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=c66ad10d7496429da3b81648cf9e83f3) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:53.399910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.401979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.409709 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 345/681] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/role_assignments?user.id=c66ad10d7496429da3b81648cf9e83f3&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:53.411050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.432200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.480537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3b07-8340-4725-8c40-b1bc89cfb462 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.534671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b1205-fa3a-4868-a7a7-275f62fc11d0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.551274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.551919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.552661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.552661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.553243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:53.553552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.555089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.576702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.578980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.579488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.579972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.580864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:53.581364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.583982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.593029 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 337/682] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:53.598542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 346/683] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:53.598989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.617646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:53.708915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.741022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:53.746036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf770b2-4d13-461f-9954-26f952b25414 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.777337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12fde2-a136-451d-99da-1cddf3b0de4e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:53.789471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.790393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.790868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.791144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.820442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:53.821069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.821352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.821624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.822337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b98deaef6c24ed2aa7c7ecc90c75205) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:53.822676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.824093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e20a3-c49f-4f8a-8502-cd7c46138543 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.830860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 347/684] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/role_assignments?user.id=0b98deaef6c24ed2aa7c7ecc90c75205&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:53.832620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143ef11-4b86-44fc-8a28-6c56b947f099 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.832907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143ef11-4b86-44fc-8a28-6c56b947f099 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.833182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143ef11-4b86-44fc-8a28-6c56b947f099 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.834673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 348/685] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:53.840059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:53.840433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:53.840775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:53.862945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:53.865413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:53.881516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:53.883612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:53.962645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 349/686] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:54.028133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.061239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 338/687] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 20:33:54.067529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.146290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.182466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98aeab-3d8e-4632-93e9-8f09168c3dc8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.239311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.240439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.240919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.241425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.260075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.269188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:54.286904 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:54.288063 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:54.288950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:54.288950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:54.288950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:54.288950 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 350/688] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:54.298848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.412252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b326ff8-7afc-4247-9815-e0ac89a669ca None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.430909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.455054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.455732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.456492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.456886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.458232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:54.458899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.461882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4a53c-5852-4535-9b21-1d0d3a35bc86 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:54.482118 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 339/689] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:54.483480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e740e1-4256-497f-9545-c39d8e4efcf1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.483765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e740e1-4256-497f-9545-c39d8e4efcf1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.484036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e740e1-4256-497f-9545-c39d8e4efcf1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.488567 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 340/690] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:54.489821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e740e1-4256-497f-9545-c39d8e4efcf1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.541119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.587748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb29b6-6f63-4522-8c94-19ec00ec5cfd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.615139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e740e1-4256-497f-9545-c39d8e4efcf1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.616229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.632002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.633097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.633553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.633998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.652057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.654346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:54.657475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.658194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.658563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.658902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.671283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.672951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-234a4c3a-c58d-4a94-bb8d-dea81b31e8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:54.673991 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:54.674946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:54.675884 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:54.675884 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:54.675884 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:54.675884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 351/691] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:54.677684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.677944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.678283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:54.692344 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:54.693245 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:54.694214 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:54.700636 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 341/692] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:54.700636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.707080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:54.708634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:54.789577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 352/693] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:33:54.790809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.840742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.845159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e481d-9210-4212-bf63-8e1a62f7685b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.891826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.892690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.893088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.893482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.894306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:54.894755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.897698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:54.923152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbba62-e0cc-445e-a99d-7bf259ad180d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:54.925344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 342/694] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:54.927062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:54.935500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:54.993298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0770-9145-472a-82d4-876886d84474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:54.993994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0770-9145-472a-82d4-876886d84474 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:54.994164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0770-9145-472a-82d4-876886d84474 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:54.994435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0770-9145-472a-82d4-876886d84474 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:54.995142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0770-9145-472a-82d4-876886d84474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:54.995440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0770-9145-472a-82d4-876886d84474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:54.997066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0770-9145-472a-82d4-876886d84474 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:55.026066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 353/695] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:55.031766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0770-9145-472a-82d4-876886d84474 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:55.150466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17a99f-68a5-4359-b23f-24ac0691f631 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:55.184402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0770-9145-472a-82d4-876886d84474 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:55.218777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:55.219484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:55.219891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:55.220169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:55.220806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:55.223142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:55.225856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:55.226558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:55.226843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:55.227111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:55.227822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b5a7880d6ec47f381af9ef1f09a767c) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:55.228181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:55.229580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:55.235988 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 354/696] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:55 2022] GET /identity/v3/role_assignments?user.id=0b5a7880d6ec47f381af9ef1f09a767c&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:55.236798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:55.260149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 343/697] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:33:54 2022] POST /identity/v3/users => generated 254 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:33:55.286977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:55.381909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90d03-cb61-432d-b24a-b1bd3cf84749 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:55.423635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:55.424285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:55.424560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:55.424828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:55.440084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:55.446536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:55.447720 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:55.448629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 355/698] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:55.456041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:55.495477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:55.602710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11954fd5-b67a-41a4-8bcd-5ba22e432d1a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:55.648159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:55.648781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:55.649106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:55.649383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:55.650130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:55.650356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:55.651789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:55.662107 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 344/699] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:55.667644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:55.790928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:55.810859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f2b-f61a-42b3-b111-016cf29e4a0b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:55.857672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:55.858302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:55.858599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:55.858877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:55.878245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:55.880008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:55.899238 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:55.900184 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:55.901676 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:55.901676 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:55.901676 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:55.901676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 356/700] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:55.905928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:56.024944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:56.062718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318d5dc-c947-4a4f-8439-525fda559f7f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:56.101886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:56.102582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.102865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.103139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.103781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:56.104081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:56.105464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:56.117990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 345/701] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:56.123480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:56.233429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1faf6-b8fd-4d59-9bec-6afa09f74bc6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:56.268424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:56.269068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.269390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.269680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.278014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:56.279499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:56.282134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:33:56.292370 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:56.293324 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:56.294771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 357/702] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:33:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:56.299557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:56.440692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32060a-1dea-46e7-8e86-7dd4a6bc4786 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:56.477323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:56.477989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.478321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.478627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.479409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:56.479714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:56.481263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ee3cc-bb29-4be6-813f-da03db448443 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:56.490169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:56.501229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 346/703] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:33:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:56.506988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:56.646364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f696009-af3d-4c68-9853-9c09bed7b931 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:56.692279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:56.692815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.693094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.693363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.786560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ab69-7923-4ac6-bab2-7dc2425ea433 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.786912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ab69-7923-4ac6-bab2-7dc2425ea433 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.787184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2ab69-7923-4ac6-bab2-7dc2425ea433 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.788805 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 347/704] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:56.794058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:56.794341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:56.794650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:56.796166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:56.799601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:56.802081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039e07-0e53-4236-805b-74080aa6a37d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:56.813387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:56.814644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:56.945928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 348/705] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:56.985658 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 358/706] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:33:56.991030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.090977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6233ac-144c-4420-8528-d0351fbaba84 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.090977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6233ac-144c-4420-8528-d0351fbaba84 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.091271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6233ac-144c-4420-8528-d0351fbaba84 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.092759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 359/707] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:33:57.099788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.100077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.100346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.129492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e8b24-11c2-4411-bc4b-63c6c2ec9edb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.135516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:57.136725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:57.171244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:57.188902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.190516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.191235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.192797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.208329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.210245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:57.227570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:57.228613 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:57.229582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:57.229582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:57.229582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:57.231691 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 349/708] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:57.243502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.286028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 360/709] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:57.295928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.345231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:57.408458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b4539-f4f3-428b-b61a-d974c6c2afa3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.475910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.476434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.476712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.476984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.477625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:57.477926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.479326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.492199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494fcbf-91de-4108-a5e9-21c4d1e32624 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.503193 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 350/710] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:57.511619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.555852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.556490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.556765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.557033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.557609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.559003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.573135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:57.582662 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:57.583719 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:33:57.584547 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:57.587796 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 361/711] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:57.589349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.687764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648495ac-dca7-41a7-b346-536edb18b3aa None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.730797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:57.734131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.734798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.735078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.735347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.742985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c1b85-0eeb-421b-b7d9-0991601181f7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.746328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.748170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:33:57.761623 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:57.762612 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:33:57.763489 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:57.766077 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 351/712] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:57.767335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.780445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.781073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.781351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.781622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.782263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:57.783035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.784101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.789545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 362/713] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:57.790238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.893767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:57.900548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4419-4c9a-4997-a85d-e6e6b23b9f33 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.901972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6af636-3abf-43b8-8e0f-cc033b7b4948 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:57.939358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.939933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.940217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.940488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.941875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:57.942312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.943766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.947122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:57.948119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:57.948610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:57.949087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] PATH_INFO: `/v3/services/network` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:57.949867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:57.952051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:57.956640 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 363/714] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:57.957592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:57.967223 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:57.968236 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:33:57.969049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:57.973108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 352/715] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:57.978210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:57.991776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:58.103968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719866e-c7dc-4dfe-9048-74ecf785aa7c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:58.107075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a935-330a-45d8-9616-336e147f09e3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:58.145640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:58.146225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.146522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.146794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.147824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:58.148136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:58.149572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:58.150149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:58.151631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.152134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.153155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.154268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:58.154888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:58.155843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 353/716] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:58.156702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501f1f9-22d2-4853-b615-a95525fd9d9b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:58.156811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:58.164996 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 364/717] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:33:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:58.216467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:58.299934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57beba-18f0-49b0-8e33-b5a2be2a3165 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.301949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57beba-18f0-49b0-8e33-b5a2be2a3165 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.302465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57beba-18f0-49b0-8e33-b5a2be2a3165 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.305234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 365/718] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:33:58.310758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc64e4-6710-48b0-bd8d-4f978e2fe86c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.311069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc64e4-6710-48b0-bd8d-4f978e2fe86c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.311565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc64e4-6710-48b0-bd8d-4f978e2fe86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.335327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf952c-e753-4100-a53e-62dc33c3328b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:58.349825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-04bc64e4-6710-48b0-bd8d-4f978e2fe86c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:58.351125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-04bc64e4-6710-48b0-bd8d-4f978e2fe86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:58.415706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:58.416397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.416600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.416869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.417510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] RBAC: Adding query filter params (name=network) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:58.417811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:58.419232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:58.424984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 354/719] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:33:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:58.432335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:58.468673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 366/720] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:58.501684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5e83-3cdc-4fe7-9477-7d82382ba850 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.502116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5e83-3cdc-4fe7-9477-7d82382ba850 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.502519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc5e83-3cdc-4fe7-9477-7d82382ba850 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.504292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 367/721] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:33:58.509397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.509680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.509950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.530684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:33:58.531653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:33:58.540393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:58.564166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:58.576941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47056-a0e7-45d0-9d80-51e6acb79054 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:58.617035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:58.617704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:58.618151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:58.618334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:58.618979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] RBAC: Adding query filter params (type=network) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:58.619337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:58.620664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:58.626884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 368/722] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 20:33:58.631500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 355/723] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:33:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:58.842156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:58.981979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc186b2-e7c6-4630-89a2-05840fe9668c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.018916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.020176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.020651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.021114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.030181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.035185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:59.047911 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:59.048867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:33:59.049700 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:59.055601 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 369/724] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:33:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:59.056588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.060170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:59.209533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6aa6e-0118-4706-b327-ac6d4dc5cbcc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.211396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002d104-3046-474e-b999-258f6501a0fd None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.251992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.252628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.252910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.253898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.272075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:33:59.278377 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:59.279384 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:59.280598 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 356/725] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:59.286052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.291200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.292066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.292559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.293020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.293932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:59.294426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:59.296051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:59.304533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:59.318003 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 370/726] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:59.324712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.417155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb9236-b5c8-4fdb-8974-ddcf1916eacc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.442115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:59.484393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.484914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.485191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.485458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.486111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:59.486407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:59.487878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:59.508914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 357/727] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:33:59.514454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.515104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5137e-9757-4848-be10-ae5c4e6ff2da None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.555175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.555807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.556087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.556361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.568771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:59.570307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:59.587220 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:59.588209 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:59.589685 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:59.589685 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:59.589685 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:33:59.589685 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 371/728] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:33:59.593939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.680557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:59.712497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8a1e3-15a8-42ca-8f3d-324251882ac9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.732751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:33:59.740113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df4ed-4a54-4690-8898-45704c1d569b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:33:59.751885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.752502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.752783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.753055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.753855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:33:59.754296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:59.755722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:59.769399 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 372/729] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:33:59.777102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.813134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:33:59.813765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:33:59.814042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:33:59.814314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:33:59.833374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:33:59.834878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:33:59.863415 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:33:59.864587 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:33:59.865975 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:33:59.865975 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:33:59.865975 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:33:59.869904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 358/730] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:33:59.871780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:33:59.959728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e006dbd-e324-432b-985a-d65cfc22c101 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.021752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.022599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.022881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.023154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.035511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.037047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.063440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f89a88-a3fc-4325-b3a2-f14791219b9f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:00.066754 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:00.069164 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:00.071388 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 373/731] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:00.078219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.118907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:00.136407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.137025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.137305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.137576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.138225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:00.138799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.141937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.147632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:00.272090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fef5ab-7638-4a6e-9f62-1bbbb52e8d58 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.319144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.319922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.320292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.320659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.321511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:00.321915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.323494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.349478 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 374/732] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:00.350887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.363697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 359/733] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:00.371527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.484355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e04f1a-3279-40ad-a383-af67108ff15a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.516290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5cd4c-ddc9-4b5b-99e4-928deac8e4b7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.523645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:00.548517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.549265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.549707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.550117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.558264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.560300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:00.574168 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:00.575367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:00.576491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:00.578849 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 360/734] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:00.586212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.600756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.601631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.602051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.602461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.689131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.691644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.721003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:00.748382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d287-d7fc-4fe8-a936-a01b4e0d4754 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.810100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:00.819884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 375/735] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 20:34:00.821485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.859228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.859853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.860130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.860402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.861143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:00.861451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.863482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:00.903934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 361/736] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:00.911466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:00.927522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340458-ce52-4632-9b9b-3451f9cd5ba0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:00.962472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:00.963534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:00.964012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:00.964525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:00.984280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:00.986222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:01.017610 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:01.018616 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:01.019539 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:01.019539 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:01.019539 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:01.028249 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 376/737] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:01.031722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.121876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.134273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df210e-2490-47c3-8aab-85173a225061 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.144140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.179725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.180902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.181393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.182058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.188925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76bf2f-c4db-4ec4-82b1-b871d956474f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.229788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.230820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.231373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.231854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.232763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:01.233175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.234666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88bf05-207c-423e-88ca-a31f00e33cf8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.253310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 377/738] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:01.253310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83eb9-69a3-4d6d-99a0-4a262d33e494 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.253310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83eb9-69a3-4d6d-99a0-4a262d33e494 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.253310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83eb9-69a3-4d6d-99a0-4a262d33e494 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.254576 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 378/739] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:01.255521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83eb9-69a3-4d6d-99a0-4a262d33e494 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.275265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.276729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a76f41-5914-43ab-bee4-e814a647d408 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.380223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83eb9-69a3-4d6d-99a0-4a262d33e494 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.389231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 362/740] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 20:34:01.389231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.389231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.389231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.390080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.400695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.407857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:01.409025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:01.419430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.419901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.420174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.422250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.432920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.434937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2ef51e0b-2371-4ccb-81db-61ed350d9c4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:01.447977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:01.448901 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:01.449767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 379/741] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:01.451184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3435e2-7972-4421-80f9-119e68368071 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.451184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3435e2-7972-4421-80f9-119e68368071 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.451184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3435e2-7972-4421-80f9-119e68368071 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.452535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 380/742] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:01.457384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3435e2-7972-4421-80f9-119e68368071 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.498529 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 363/743] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:34:01.498529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.584887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3435e2-7972-4421-80f9-119e68368071 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.622444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.623189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.623558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.623958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.626180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368072b-5f7c-49a5-9478-37d883d59c42 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.645575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.647458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ca9c2452-b86d-438f-ad76-4f19f6ed41a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:01.663288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:01.664300 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:01.665814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:01.665814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:01.665814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:01.665814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 381/744] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:01.666037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.666305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.666619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.676645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.677548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.678021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.678525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.679411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:01.679974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.681675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.684569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.686223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:01.687583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:01.697218 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 364/745] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:01.698399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.708013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.766620 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 382/746] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:34:01.767781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.806466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6aa2c-75f3-4f88-ac6b-76e94d322a54 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.845547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.846303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.847417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.847798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.848537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:01.848944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.850887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741daba8-1036-411a-9201-d226aeeed4ef None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.867666 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 365/747] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:01.869037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcca50-5440-4c42-bd83-42b85296dc73 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.869425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcca50-5440-4c42-bd83-42b85296dc73 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.869798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcca50-5440-4c42-bd83-42b85296dc73 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.871379 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 366/748] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:01.872170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcca50-5440-4c42-bd83-42b85296dc73 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:01.875224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b82e84-49fc-40cd-b348-eac6ba5db3c8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:01.913692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:01.914457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.914457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.914806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.915458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=f329c05a7437450db2315117edbb65ea) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:01.915756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:01.917160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179f37-b0ba-4769-bdf6-5801f2c02aa5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:01.926529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 383/749] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/role_assignments?user.id=f329c05a7437450db2315117edbb65ea&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:01.926529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:01.926529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:01.926529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:01.941423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:01.948347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:01.949584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:01.958238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.002404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcca50-5440-4c42-bd83-42b85296dc73 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.039018 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 384/750] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 20:34:02.040298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.051462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.052117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.052456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.052734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.081685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:02.086816 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:02.087798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:02.088610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:02.095673 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 367/751] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:02.096659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.166465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854f139-ab45-4103-a11e-4569716fa88b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.204338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.205327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.205813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.206237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.214249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.215919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.225844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152b10-46ec-465d-aead-b728d54746c9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:02.228378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:02.229237 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:02.230115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 385/752] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:02.230619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.233071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.252511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.278257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.279585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.280051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.280784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.281638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:02.282373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.285180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.297665 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 368/753] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:02.299165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.341740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fadc5-f764-4866-9d9c-5369e80924a6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.382196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.382855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.383196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.383546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.384237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.385725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.403804 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 386/754] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:02 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:02.405408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.426320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd3e81-7854-4a7c-bda2-5392a1057a22 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.490339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.491187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.491857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.492142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.492454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.493429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:02.493782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.495465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.518337 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 369/755] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:02.519415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.544654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1bf20-256e-4561-8872-01b3a05a32b3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.560791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.594630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.596070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.596610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.597102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.612039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.614914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:02.633458 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:02.634841 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:02.635783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:02.635783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:02.635783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:02.642055 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 387/756] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:02.643132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.656991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fff263-138f-4659-8307-403dea19d25e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.703611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.704450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.704727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.705299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.705895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.707356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.734519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 370/757] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:02 2022] POST /identity/v3/services => generated 240 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:02.734519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:02.817828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d6c04-b628-4279-8831-868763a5f177 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:02.850019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.892420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:02.893465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:02.893970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:02.894484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:02.895382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b5a7880d6ec47f381af9ef1f09a767c) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:02.895839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:02.897988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:02.905857 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 388/758] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/role_assignments?user.id=0b5a7880d6ec47f381af9ef1f09a767c&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:02.964708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:02.974929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1b9e0-b6f0-4ed7-b8c7-a353195cd25e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.029786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:03.030413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:03.030716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:03.030989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:03.031644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:03.031948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:03.033339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:03.057180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 371/759] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:03.062714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:03.192931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0951cc-4181-4f53-86b9-4165e892dc43 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.230918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:03.231532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:03.231817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:03.232092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:03.240023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:03.241512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:03.254057 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:03.255091 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:03.256024 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 389/760] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:34:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:03.261028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:03.388368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c321-ae45-4a5c-8fc7-58d0aa098ee8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.433065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:03.433730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:03.434012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:03.434284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:03.435039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:03.435338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:03.436765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:03.460537 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 372/761] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:34:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:03.465962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:03.522970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:03.589289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e882095-e1c6-44db-83a9-dfb99bde4f33 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.624543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:03.625182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:03.625463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:03.625733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:03.636846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:03.638608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:03.654140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:03.655128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:03.655990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:03.655990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:03.655990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:03.657711 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 390/762] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:03.663988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:03.680787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:03.799526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17def7-9cff-411f-adf3-24028c032d77 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.834934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:03.835541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:03.835817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:03.836090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:03.836733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:03.837031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:03.838468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:03.850461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 373/763] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:03.855978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:03.950342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:03.963645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da7038-c2be-40ce-aade-65972a35ddf4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:03.999779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:04.000419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:04.000739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:04.001021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:04.008743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:04.010200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:04.022055 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:04.022996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:04.023886 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 391/764] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:04.029112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:04.171462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:04.210609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf02f4-00ba-4b8e-ae9f-8976ff1dae91 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:04.283771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:04.285411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:04.285895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:04.286362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:04.287294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:04.287802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:04.289562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18ed303-2196-492b-84f0-22b0aacad98d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:04.317172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 374/765] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:04.323728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:04.535065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26902-7141-4246-bcfe-5e778a9b823f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:04.580085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:04.582742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:04.583024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:04.583302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:04.588200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=6531dafbedef45ccb7b6b7e2f610c2c2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:04.589772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731f70f-882b-44cf-a14d-3c9121f6c960 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:04.611185 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 392/766] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:04 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:04.980962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a1dd-5406-43c9-a371-73a64cb034b9 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:04.981489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a1dd-5406-43c9-a371-73a64cb034b9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:04.981489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555a1dd-5406-43c9-a371-73a64cb034b9 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:04.983166 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 375/767] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:04.990467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c55cd-a2ac-448d-ab89-367858419780 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:04.990797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c55cd-a2ac-448d-ab89-367858419780 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:04.991043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c55cd-a2ac-448d-ab89-367858419780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:05.005950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:05.011340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-db5c55cd-a2ac-448d-ab89-367858419780 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:05.012536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db5c55cd-a2ac-448d-ab89-367858419780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:05.084000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 393/768] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:05.457236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555a1dd-5406-43c9-a371-73a64cb034b9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:05.481602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84cd8-ca55-4af9-8ea4-5d4a9332f7d1 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:05.481949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84cd8-ca55-4af9-8ea4-5d4a9332f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:05.482146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef84cd8-ca55-4af9-8ea4-5d4a9332f7d1 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:05.483981 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 394/769] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:05.489812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:05.490097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:05.490374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:05.508832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:05.510001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:05.591662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555a1dd-5406-43c9-a371-73a64cb034b9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:05.624436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 395/770] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:05.627127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:05.639348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:05.639994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:05.640271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:05.640540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:05.658387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:05.664320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:05.665225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:34:05.666047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:05.672883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 376/771] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:34:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:05.674186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:05.765817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:05.812632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7decf-437a-4756-a4f1-2ee9faaa807a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:05.864040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:05.864751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:05.864977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:05.865186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:05.865838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:05.866138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:05.867525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:05.881541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 396/772] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:34:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:05.887158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.022890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9dafc-5bf1-4ef6-9fff-9e9f0158f8db None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.064611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.065237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.066100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.066382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.080284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.081764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0d6c0459-c970-45f3-9f72-dcc6690101fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:06.102288 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:06.103298 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:06.104165 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:06.104165 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:06.104165 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:06.104165 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 377/773] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:06.109043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.214220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e34f6-e853-496b-b245-87f4b64fe445 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.236892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6a76-6cd1-4271-bfd5-d2ab70352cb8 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.237212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6a76-6cd1-4271-bfd5-d2ab70352cb8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.238142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6a76-6cd1-4271-bfd5-d2ab70352cb8 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.242539 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 378/774] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:06.244323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.244756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.245173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.269947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.276466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:06.277973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:06.279385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.280320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.287091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.287591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.288493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:06.288948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.290793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54967d-420c-4ff7-b815-b95293b81478 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.305758 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 397/775] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:06.307319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe5-a860-45a2-b8fa-9de9d8008eee None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.307756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe5-a860-45a2-b8fa-9de9d8008eee None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.308178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883afe5-a860-45a2-b8fa-9de9d8008eee None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.309830 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 398/776] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:06.312362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883afe5-a860-45a2-b8fa-9de9d8008eee None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.348615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.378714 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 379/777] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:06.379699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.459175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883afe5-a860-45a2-b8fa-9de9d8008eee None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.510926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ff024-b96d-45e8-941e-d9bbe876b9aa None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.514228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.515184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.515666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.516136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.526986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.529275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-3cbc3fcc-e4f9-4754-b287-e522cdb26b7d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:06.547812 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:06.548880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:06.549956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:06.552402 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 399/778] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:06.553786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.554299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.554775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.561620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.562469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.562875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.563245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.564397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.566005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.577318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:06.578806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:06.579552 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:06.580668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:06.582151 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 380/779] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:06.583231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.633265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.648962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.691323 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 400/780] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:06.691323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.734146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380082c3-0b09-4f13-92f0-82c47d06a4c7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.810974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.811507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.811782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.812051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.812780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:06.813081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.814540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.846149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4219b-938c-483c-835d-6cf5b9df901a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:06.848904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 381/781] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:06.854631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.907248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:06.907865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:06.908202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:06.908474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:06.909729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:06.910195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:06.912131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:06.922609 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 401/782] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:06.923849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:06.989387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.992324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:06.997895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a3eb-818d-4a52-9c68-eeb95aec1732 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.044193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.045881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.046366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.047124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] PATH_INFO: `/v3/services/network` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.048065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.049849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:07.063454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:07.064469 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:07.065891 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:07.068899 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 382/783] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:07.070222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.080878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e30495-2140-4a67-8eb8-701448fc90e0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.124633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.125280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.125537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.125810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.139649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.141368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:07.159771 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:07.160853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:07.166692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:07.166692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:07.166692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:07.166692 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 402/784] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:07.166692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.189837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102001d1-9900-4075-88ef-ae1807660693 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.231127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.231844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.232125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.232401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.233028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.233328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.234717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.244475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 383/785] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.244475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.269419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.276576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.285690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7b937-bd5f-4985-876d-a40d4e81bb84 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.324175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.324736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.325011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.325281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.338474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:07.343853 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:07.344810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:07.345625 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 403/786] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:07.346430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.359421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b5c-be3d-496c-8a1a-d7299ab3ee5a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.396892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e32de-070d-4090-a1a4-94339908907c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.397918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e32de-070d-4090-a1a4-94339908907c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.398624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e32de-070d-4090-a1a4-94339908907c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.398906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e32de-070d-4090-a1a4-94339908907c None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.399542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e32de-070d-4090-a1a4-94339908907c None admin] RBAC: Adding query filter params (name=network) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.399909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e32de-070d-4090-a1a4-94339908907c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.401295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e32de-070d-4090-a1a4-94339908907c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.406325 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 384/787] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.406943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e32de-070d-4090-a1a4-94339908907c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.467369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe771ee1-3f90-42c7-a01d-7e83d5a0e1f1 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.502088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.502784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.503610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.503879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.504516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.504834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.506176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.506711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.523456 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 404/788] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.523456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.523744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e32de-070d-4090-a1a4-94339908907c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.539278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.558527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.559270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.559647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.560043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.560840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] RBAC: Adding query filter params (interface=public, service_id=2881ebe93c594278ae29b6cf8d4e51d8, region_id=RegionOne) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.561253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.562782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.570716 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 385/789] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/endpoints?service_id=2881ebe93c594278ae29b6cf8d4e51d8&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.571511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.636956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa77b43-b3ad-4ce2-ba7c-fa5fb3920ac8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.673621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.674393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.675819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.676193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.677268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.677669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.679130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.687884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e88a7-18da-4d64-afed-7b1b1370ae38 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.688025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 405/790] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.688899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.725302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.725905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.726181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.726445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.727099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] RBAC: Adding query filter params (type=network) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:07.727401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.728772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.734126 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 386/791] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.734930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.760385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.777223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.800880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd776d-0b6b-4d7d-8385-83121864de44 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.833807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.834428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.834732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.835011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.840439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f304f4-5793-4233-bbca-451aad404bff None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.841557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.843055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:07.854368 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:07.855449 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:07.856353 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:07.860237 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 406/792] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:07.860910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.875283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:07.875884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:07.876158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:07.876432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:07.877026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:07.878697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:07.883854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 387/793] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:07.884547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:07.975277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:07.983476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa0ac-4e89-49b1-95ff-d744cde2822b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:07.998427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd0ad3-fd31-4459-9885-16a650b7fe6f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.025466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.026301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.026628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.027282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.030332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:08.040675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.041083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.041301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.041580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.041851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.042480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.042696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] RBAC: Adding query filter params (interface=public, service_id=d3d8e135aa874eb3b4db86dbd177da22, region_id=RegionOne) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:08.043013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.044387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a3f4e-3df7-4fe0-b924-3d1cc39d032d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 388/794] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/endpoints?service_id=d3d8e135aa874eb3b4db86dbd177da22&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138eb8-81f6-4fce-8cca-fb4570e37e03 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138eb8-81f6-4fce-8cca-fb4570e37e03 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02138eb8-81f6-4fce-8cca-fb4570e37e03 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 389/795] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:08.055796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02138eb8-81f6-4fce-8cca-fb4570e37e03 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:08.069213 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:08.070483 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:08.071615 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:08.071615 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:08.071615 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:08.074542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 407/796] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:08.075657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.177628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02138eb8-81f6-4fce-8cca-fb4570e37e03 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.195902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b2e24-fa3f-4f13-ae26-9e57f35f7850 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.214810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.215430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.215696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.215970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.216786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:08.217093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.218549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b6022-9a5b-4228-845f-2f420ebc0978 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.236707 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 390/797] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:08.236914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.236914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.236914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.237177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.237286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.237432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.237432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.237600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.237600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.242461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:08.243041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 408/798] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:08.243863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.270463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:08.272243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:08.321181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:08.359138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 391/799] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:08.360570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.390291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee8530-8836-4aa0-8098-d8c03e050cba None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.435642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.436978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.437268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.437544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.438220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:08.438560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.440332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.460227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 409/800] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:08.466714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.531486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55088cc0-9905-497e-8633-069a5588f4cc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.578737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.579825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.580133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.580413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/f329c05a7437450db2315117edbb65ea/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.583473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:08.597822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cf69d-6f57-41a5-b100-294393f1d15e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.601981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=f329c05a7437450db2315117edbb65ea, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.603464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.616257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:08.623212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 392/801] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:08 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/f329c05a7437450db2315117edbb65ea/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:08.654330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.655551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.656050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.656521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.666613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.668508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:08.682992 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:08.684453 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:08.685413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:08.688066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 410/802] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:08.694301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.826018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436787-df99-45fc-b500-b26396aa56a1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.827139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.890660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:08.891717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:08.892212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:08.892683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:08.893646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:08.894166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:08.896359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:08.920254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 393/803] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:08.926735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:08.964347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920177a-5682-4416-9dd0-fcaabd1d3076 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:08.976751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.000633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.001349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.001694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.002034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.015929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:09.022294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:09.023288 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:09.024148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 411/804] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:09.036385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.057425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f65b80-2fae-4034-b912-3cc98a60baa5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.099790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.105118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.106780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.107215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.107652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.119541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.122012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:09.138856 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:09.140042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:09.141151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:09.141151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:09.141151 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:09.143568 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 394/805] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:09.150107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.269153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974d650-edb9-49f3-a1bf-19ddf8f4a2d4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.293150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.301356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6f055-34b7-493c-bd08-f1b7b38c429a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.340129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.340380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.340741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.341024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.341297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.341875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.341981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:09.342245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.342678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.343149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.343638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.344188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:09.344676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.346859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.357077 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 412/806] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:09.363702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.371812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 395/807] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:09.378175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.483120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.485985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee114e61-64f6-4c44-ba37-654cf2cd7578 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.522770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.523328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.523610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.523884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.534561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.536016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:09.552441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:09.553414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:09.554997 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:09.554997 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:09.554997 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:09.554997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 413/808] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:09.559623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.608432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be514a6-61e3-42a6-b828-77144a8be64d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.676089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9d4d3-6617-4326-9825-109658fd5051 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.711571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.712310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.712511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.712781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.713473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:09.713781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.715287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.729271 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 414/809] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:09.735278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.758860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.759204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.759853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.760131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.760404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.787403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.788786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:09.821129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:09.822077 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:09.823036 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:09.833956 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 396/810] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:09.838341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:09.872553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:09.873063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7a31b-0bf7-4621-8aac-51c2578e1c92 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:09.914669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:09.915294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:09.915573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:09.915841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:09.933929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:09.935405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:09.954562 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:09.955580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:09.957116 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:09.961448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 415/811] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:09.964789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:10.015017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993f78-e4a1-465d-9d3c-089f951b3137 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:10.055467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:10.055998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.056272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.056538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.057292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:10.057595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:10.059062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:10.078647 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 397/812] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:10.081659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:10.151410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:10.215058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:10.229131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a94a-7bf3-4439-8df7-9a297dd717d5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:10.259990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a23c7d-6f89-402b-9570-7ecca2e3ec58 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:10.307675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:10.309569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.311046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.311593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b5a7880d6ec47f381af9ef1f09a767c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.314147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:10.317429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=0b5a7880d6ec47f381af9ef1f09a767c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:10.319616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.319616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.319616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.319616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:10.319616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:10.321650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:10.323058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:10.348434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 398/813] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:10 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/0b5a7880d6ec47f381af9ef1f09a767c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:10.353103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 416/814] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:34:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:10.358654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:10.502069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8485fd8-d51b-4b8c-9c26-2faabd494e05 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:10.542590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:10.543310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.543650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.543925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.546781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:10.641674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:10.643771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db3b99-adf3-4454-b01e-c3084a757daa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:10.757415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 399/815] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:10.762777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:10.766995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ea1c-0928-410c-a66f-f0d52aa779c5 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.767350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ea1c-0928-410c-a66f-f0d52aa779c5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.767692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ea1c-0928-410c-a66f-f0d52aa779c5 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.770073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 400/816] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:10.781337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.781686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.782021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.816313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:10.817572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:10.883560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:10.887226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f73e92-4ec8-4cc9-ac01-d48f10c122cc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:10.913673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:10.934985 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 401/817] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:10.943257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:10.943965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:10.944251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:10.944526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:10.958447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:10.959933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:10.986566 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:10.988098 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:10.989431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:10.989431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:10.989431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:10.989431 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 417/818] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:10.993415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.142635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471571a-e8cc-4688-af63-64ee40a4c0a6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.194128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.194776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.195056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.195328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.195979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:11.196278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.197656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.210959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 402/819] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:11.216466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.335705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.391619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:11.397227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3199e-55c6-4e62-8482-09ad98cae4d5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.444326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.444956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.445231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.445556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.453251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.454708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a0d604a7-7d18-415c-b8cc-41483d32464a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:11.473746 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:11.474726 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:11.475591 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 418/820] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:11.476512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b7e7-cff2-4539-ac61-c14399497b5c None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.479334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b7e7-cff2-4539-ac61-c14399497b5c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.479334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b7e7-cff2-4539-ac61-c14399497b5c None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.479334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 419/821] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:11.483043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b7e7-cff2-4539-ac61-c14399497b5c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.491622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60063b-420b-43e0-a84e-149993292ab9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.521832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:11.527585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.528223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.528500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.528772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.529356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.530763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3d646cf3-595a-4f7f-b230-1aba660b08c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:11.540441 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:11.541664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:11.542687 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 403/822] 10.222.0.52 () {56 vars in 1129 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:11.543493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.543493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.543627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.562736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:11.564009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:11.606190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b7e7-cff2-4539-ac61-c14399497b5c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.638577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 404/823] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:11.639871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.652027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.652569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.652849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.653123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.653811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:11.654116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.655577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab78d4a-6796-4386-a177-f80b8d22de67 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.668403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 420/824] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:11.669724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383019e3-f946-4895-aaaf-fcf561c59bbc None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.670021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383019e3-f946-4895-aaaf-fcf561c59bbc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.670299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383019e3-f946-4895-aaaf-fcf561c59bbc None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.676427 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 421/825] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:11.676427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383019e3-f946-4895-aaaf-fcf561c59bbc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.677166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:11.779798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:11.788008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383019e3-f946-4895-aaaf-fcf561c59bbc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.796031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4d1d6-0a5d-4f96-88ae-766bc44b8fa3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:11.825249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.825847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.826122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.826390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.827152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:11.827458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.828868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fab4fd-b965-4ab9-b43b-13e326b35a62 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.836024 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 422/826] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:11.837276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.837563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.837837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.844289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:11.846965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:11.847781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:11.848351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:11.849228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:11.849805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:11.851537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:11.857600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:11.858659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 405/827] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:11.858849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:11.870527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:11.948046 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 423/828] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:12.026988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.053539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c709ff-3ba6-4ccc-9c72-c8e190f19ac5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.060338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.098955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.099586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.099861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.100129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.100767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:12.101065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.102468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.115335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 406/829] 10.222.0.52 () {56 vars in 1124 bytes} [Sat Sep 24 20:34:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:12.127745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.191561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.199840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d811f58-3319-41c2-b1f3-f79ddd2b85b7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.243691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.244358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.244635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.244906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] PATH_INFO: `/v3/services/network` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.245577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.247137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:12.257304 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:12.258295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:12.259276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 424/830] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:12.269701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.271581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110cf0b-6931-4e52-89ad-30d1851872bc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.308616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.309657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.310228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.310739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.311594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.313155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.339879 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 407/831] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:12.341107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.344104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.443331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe42b-7a15-4fd6-8862-644f5f48bfbd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.487285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80376f-be17-434f-b30b-de30f8b5ac2a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.524514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.525141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.525418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.525686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.526320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] RBAC: Adding query filter params (name=network) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:12.526678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.529001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.535259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 425/832] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:12.541027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.557381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.558090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.558370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.558678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.585443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:12.586977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:12.596334 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:12.597674 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:34:12.598483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:12.608982 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 408/833] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:12.612993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.691241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.745239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b750-e1be-4d6c-bdde-8d45149be06b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.754806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4586665-f43d-4ae4-a8cd-ab99dd5003dd None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.800733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.801345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.801621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.801884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.802572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:12.802867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.804226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.805338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.805943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.806214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.806474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.807118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] RBAC: Adding query filter params (type=network) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:12.807412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.808740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:12.819132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 426/834] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:12.822438 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 409/835] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:12.825065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.829023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:12.918701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:12.931837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66896963-5d5b-4dc1-82a8-4930813ff67b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:12.973094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:12.973720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:12.973994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:12.974264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:12.974956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:12.976399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:13.008840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 427/836] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:13.054749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07736-b9c5-457f-a642-e4c7b435df02 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:13.087053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:13.120904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:13.121547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.121821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.122087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.137343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:13.138850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:13.160518 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:13.161497 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:13.163014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:13.163014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:13.163014 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:13.163014 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 410/837] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:13.167365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:13.324881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed574340-6b20-4793-8e15-d350683885cf None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:13.370036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:13.370703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.370985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.371258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.371902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:13.372204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:13.373572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7617d-398e-43cc-a307-01f5c40e3fb4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:13.383655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:13.392003 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 428/838] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:13.398268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:13.464426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549f0d-9be9-4fba-bf9f-4ead4f3f87c8 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.464749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549f0d-9be9-4fba-bf9f-4ead4f3f87c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.464979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549f0d-9be9-4fba-bf9f-4ead4f3f87c8 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.466308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 429/839] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:13.471865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.472163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.472522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.501256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:13.502729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:13.531207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa354ed-c129-4c04-a7dc-6aa71d58ee23 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:13.591094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:13.592319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.592862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.593329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.605365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:13.608105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:13.612156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 430/840] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:13.622991 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:13.623892 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:13.624899 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:13.627481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 411/841] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:34:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:13.642651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:13.747449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684cd7c-3698-4807-9170-3e246f520dfa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:13.776053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:13.785744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:13.786657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.787133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.787620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.788534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:13.789036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:13.790658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:13.809511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 431/842] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:13.816611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:13.845846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:13.925507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e9174-804f-4bf4-8c19-aa4897877c84 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:13.960982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:13.961770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:13.962047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:13.962315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:13.984014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.038424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.040092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.068933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.096545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad90-a5b3-4764-8092-b479a086063c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.132459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.133091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.133368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.133638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.145050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:14.150068 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:14.151208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:14.152054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 432/843] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:14.156850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.164113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 412/844] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:14.169862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.223025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.276204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dd74a-9b24-445d-ace2-b75cf48a437c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.295911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d1f98-b30f-4f38-8892-7cb59c19dc63 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.307145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.316724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.317635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.318117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.318636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.329128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.331441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.342301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.342959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.343237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.343511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.344184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:14.344490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.346127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:14.351336 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:14.352317 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:14.353212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:14.353212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:14.353212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:14.355515 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 413/845] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:14.358672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 433/846] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:14.363644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.371633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.500098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae72ee-8f87-4b17-b56e-c0fc048f1a2a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.506033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a5f45-2dfd-4fa1-bb63-3c86d96b1025 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.513554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.542264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.542904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.543178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.543449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.544084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:14.544380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.545883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.551725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.552617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.553137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.553625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.563825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 414/847] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:14.565324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.571412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.572939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:14.588915 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:14.589885 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:14.590810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:14.590810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:14.590810 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:14.592853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 434/848] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:14.602609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.618876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.704910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002729-a6d4-4161-8e65-b37907d1b24c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.713299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255267b-285a-4d0d-94e9-f88b3cfc350b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.754920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.755659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.756001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.756338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.757395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.758534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.758996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.759846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.760723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:14.761224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.762792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.764140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.765642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:14.778234 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:14.779319 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:14.780246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:14.781292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 435/849] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:14.786546 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 415/850] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:14.787642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.789883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.812910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.901413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5777f5-c8dd-488e-877d-e8ed9decde50 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.904442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:14.907930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ed6c1-c87b-4da3-bd56-a0ae4a8358ad None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:14.938293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.939211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.939488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.940492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] PATH_INFO: `/v3/users/swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.948017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.948892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:14.949445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.949547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.949754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.950020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.950729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:14.951029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:14.952478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e8e5f-455a-4b0d-b59b-9952472ebf20 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-8a084606-4ceb-4bcb-8d78-255bd45fd776 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:14.961481 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:14.962437 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 436/851] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8276306-b39b-4d4b-8b07-3ed2cc2d1a12 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.967734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8276306-b39b-4d4b-8b07-3ed2cc2d1a12 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.968459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8276306-b39b-4d4b-8b07-3ed2cc2d1a12 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.968459 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 437/852] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:14.970226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8276306-b39b-4d4b-8b07-3ed2cc2d1a12 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:14.976540 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 416/853] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:14.976540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:14.976540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:14.976540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:14.991904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:14.993096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:15.060824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.069712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 417/854] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:15.069712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.093117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8276306-b39b-4d4b-8b07-3ed2cc2d1a12 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.136242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.136868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.137145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.137476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.138204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:15.138786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.140877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.165078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 438/855] 10.222.0.52 () {56 vars in 1135 bytes} [Sat Sep 24 20:34:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:15.170721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.171680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.183328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f03eb-762d-402a-9955-8045d90d78e7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.221717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.222538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.222899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.223233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.224102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=f329c05a7437450db2315117edbb65ea) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:15.224404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.225844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd942d-ca08-41a8-a26d-c7c85a138f4c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.232657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 418/856] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/role_assignments?user.id=f329c05a7437450db2315117edbb65ea&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:15.265500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88200-6da5-48f5-9ee2-f4310f3d924a None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.265837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88200-6da5-48f5-9ee2-f4310f3d924a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.266048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88200-6da5-48f5-9ee2-f4310f3d924a None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.267429 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 419/857] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:15.275099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.275433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.275646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.297819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:15.299186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:15.307744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7353-3004-4db2-8409-e863baa0cd4d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.389534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.390364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.390744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.391011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.410891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.463712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 420/858] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:15.463712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.517545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.519841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.586811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.612573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b3396-9fd8-48d8-88bc-f5b7fed4785b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.657271 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 439/859] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:15.662530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.673492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.675393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.676239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.676713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.677735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.680439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:15.693421 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:15.694374 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:15.695571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:15.696289 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 421/860] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:15.700273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.719467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.794942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed772-feef-4f0b-8e50-507d5d41b0b4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.815767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5314c6f-5927-44b6-a574-fb04e43affb1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.833941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.834618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.834885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.835157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.845800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.847558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.852365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:15.853197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:15.853654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:15.854101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:15.854993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:15.855474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:15.857187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:15.866540 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:15.867474 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:15.868340 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:15.868340 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:15.868340 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:15.868509 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 422/861] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:15.868595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 440/862] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:15.868671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.869697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:15.884611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.975667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:15.979289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5507ab-cbba-4996-ac7a-d7f1ff0cb3e5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:15.987833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e44a90-a356-4388-be6b-b7c0da357f5a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.014185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.014842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.015126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.015391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.016033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.017401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.024854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.025452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.025728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.025997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.026685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.026972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:16.027153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:16.028090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:16.031610 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:16.032336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.032408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 423/863] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:16.032408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.044832 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 441/864] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:16.045651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.126792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:16.151048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de6005-64b8-4828-974d-fe729542e20d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.152958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be458ce4-96b4-47db-aa5f-e62d70f15bc9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.187043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.188046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.188333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.188603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.189225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.189756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.191164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.191287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.191768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.192045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.192501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.193420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.193725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.195133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.200334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 442/865] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:16.200334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.200561 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 424/866] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:16.201202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.229466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:16.330091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cbc55-fde4-49df-9a78-54b2375ec5ee None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.363167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b591438-126c-4ac9-9dbd-488490f5bb77 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.389012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.389657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.389931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.390198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.400249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.401716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.409853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.410475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.410777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.411052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.411693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.411999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.413400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:16.417072 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:16.417956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:16.419053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 443/867] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:16.423481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 425/868] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:16.424479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.523820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:16.554847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5fa6b-8885-494c-9249-1537ad0349dc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.593811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.594432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.594741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.595011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.595759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.596573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.598024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac367e-a58f-49d7-a614-6dd587b0ef12 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.617131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 444/869] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:16.623128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:16.643510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:16.750657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6effa-936c-42f1-8756-214f05d05a50 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:16.788738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:16.789694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:16.790187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:16.790687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:16.791629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=0b5a7880d6ec47f381af9ef1f09a767c) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:16.792139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:16.793852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0facb-10a3-4b64-9795-1b48f4f72eb5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:16.800626 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 426/870] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:16 2022] GET /identity/v3/role_assignments?user.id=0b5a7880d6ec47f381af9ef1f09a767c&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:17.879001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457849d-5518-4ac2-b18a-ddd8c514208d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:17.879517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457849d-5518-4ac2-b18a-ddd8c514208d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:17.879517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457849d-5518-4ac2-b18a-ddd8c514208d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:17.881086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 445/871] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:17.886387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:17.886816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:17.887047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:17.905866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:17.907071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:17.974571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:17.987512 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 427/872] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:18.322343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457849d-5518-4ac2-b18a-ddd8c514208d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:18.437879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457849d-5518-4ac2-b18a-ddd8c514208d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:18.475244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:18.475898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.476176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.476446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.478634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:18.490519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:18.495690 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:18.496689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:18.497559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 446/873] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:18.503860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:18.614662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a320f4-01c9-48cd-b843-f52c08850759 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:18.662821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:18.663812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.664297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.664800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.665659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:18.666190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:18.668057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0cdf6-70b3-4ae4-a58c-50754389f68f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:18.679378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 428/874] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:18.684726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:18.791046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c09aa-c2e3-462a-808d-06fdbbff066e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:18.805890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83819-c7f0-4bbf-bfa2-3bb152fae766 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.806524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83819-c7f0-4bbf-bfa2-3bb152fae766 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.806905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a83819-c7f0-4bbf-bfa2-3bb152fae766 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.808703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 429/875] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:18.813821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266a03-892c-490b-bf98-d8dcb513d738 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.814405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266a03-892c-490b-bf98-d8dcb513d738 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.815622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9266a03-892c-490b-bf98-d8dcb513d738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.827426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:18.828052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.828331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.828601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.836598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b9266a03-892c-490b-bf98-d8dcb513d738 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:18.838160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9266a03-892c-490b-bf98-d8dcb513d738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:18.839201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:18.840919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:18.859503 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:18.860522 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:18.861974 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:18.861974 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:18.861974 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:18.861974 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 447/876] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:18.868340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:18.921920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:18.927102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:18.942984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 430/877] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:18.964214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6901e3-3dba-41b5-a147-5d48580f06e9 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.964532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6901e3-3dba-41b5-a147-5d48580f06e9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.964802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6901e3-3dba-41b5-a147-5d48580f06e9 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.969977 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 431/878] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:18.972235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:18.972595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:18.972946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:18.991242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:18.992537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:18.996604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ca2aa-7200-47eb-b577-b70ed3301824 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.035489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.036157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.036502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.036827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.037526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:19.037835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.039236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.053986 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 448/879] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:19.063438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.078644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 432/880] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:19.178066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87deda-7c06-47d5-af58-849a276254dd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.185549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:19.214635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.215267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.215543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.215812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.223174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.224711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:19.236574 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:19.237693 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:19.238633 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 449/881] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:34:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:19.245604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.273597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.354890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff79cf-fa3c-4547-9737-63747c31d4de None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.378754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:19.393565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8825584-31d6-46e7-b79a-87114f65d4c8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.398695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.399301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.399577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.399841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.400555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:19.400854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.402277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421092fc-2120-4088-90c7-8bd1d28739cf None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.426585 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 433/882] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:19.430254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.430935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.431318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.431711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.432243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d25b6-3798-4a51-bad9-7f3545b5a93d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.432509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.432600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d25b6-3798-4a51-bad9-7f3545b5a93d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.432785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d25b6-3798-4a51-bad9-7f3545b5a93d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.434055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.434204 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 434/883] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:19.435071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d25b6-3798-4a51-bad9-7f3545b5a93d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.451522 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 450/884] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:19 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:19.452419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.482827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:19.565618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d25b6-3798-4a51-bad9-7f3545b5a93d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.573094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad5c04-3467-4fac-aaaf-382fa37db984 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:19.603429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.604052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.604333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.604605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.605273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.606727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cec90-2f01-445b-a382-a98b83e18e89 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.613005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:19.613919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.614267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.614612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.634958 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 435/885] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:19 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:19.636491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:19.636777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:19.637047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:19.675877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:19.677136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:19.711341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:19.730180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:19.731707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:19.777126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:19.786586 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 436/886] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:19.868752 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 451/887] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:19 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:19.874792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:19.984079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc7a22-d1fe-4bee-8203-9ede5bd14c41 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.022798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.023794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.024278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.024789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.040764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.040764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:20.057260 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:20.058228 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:20.059135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:20.059135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:20.059135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:20.059135 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 437/888] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:20.066206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.130605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.152136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:20.177224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc0c4-5832-415b-8b33-997170120d3e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.204697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:20.216430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.217037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.217334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.217616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.218219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:20.218549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.219915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.232773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 452/889] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:20.239122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.272005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e2ad-cd25-4d46-afb9-0a8fde37d16a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.308207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.309085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.309485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.309918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.310639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.312123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:20.321472 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:20.322682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:20.323554 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:20.326533 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 438/890] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:20.336326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.350104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96273-a64e-4a19-8394-e43ac0e88c0e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.384967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.385596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.385873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.386145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.393712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.395261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:20.407202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:20.408202 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:20.409441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 453/891] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:20.415442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.426327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:20.457511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68805a-c32e-4d2c-b68a-cea5aa073daa None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.482891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:20.495264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.496070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.496516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.496939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.497704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:20.498148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.499801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.507983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 439/892] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:20.513544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.553416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9410d-b0c1-4a1b-bd90-c46271bfeb21 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.603270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.604839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.605357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.606062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.606974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:20.607656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.610168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.634900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 454/893] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:20.639280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:20.651680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235e44-ea69-41f8-8763-d1be7d01c636 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.688142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.688886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.689805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.690173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.690969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:20.691348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.692886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcafd9dd-7d6a-43da-b528-5989bdde356a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.702521 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 440/894] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:20.772896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba776b5-ffab-4e04-ac6d-0c85e5286e21 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:20.781428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:20.816759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:20.817767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:20.818225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:20.818679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:20.819584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] RBAC: Adding query filter params (role.id=ab8b8d0424e74f6cb1092f08abf63252, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=f329c05a7437450db2315117edbb65ea) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:20.820051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:20.821727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd45586-31e0-48ad-bfd1-a8504bd8ddfd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:20.829019 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 455/895] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:20 2022] GET /identity/v3/role_assignments?user.id=f329c05a7437450db2315117edbb65ea&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=ab8b8d0424e74f6cb1092f08abf63252 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:21.989380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb7bc-e7fa-423a-bf51-bc7e765f6fc0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:21.989878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb7bc-e7fa-423a-bf51-bc7e765f6fc0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:21.989878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb7bc-e7fa-423a-bf51-bc7e765f6fc0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:21.991451 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 441/896] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:21.996677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527542b0-cedf-4de3-9a67-48cd66283c1d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:21.997093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527542b0-cedf-4de3-9a67-48cd66283c1d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:21.997267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527542b0-cedf-4de3-9a67-48cd66283c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.016296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-527542b0-cedf-4de3-9a67-48cd66283c1d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:22.018152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-527542b0-cedf-4de3-9a67-48cd66283c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:22.094321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 456/897] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:22.172631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a020c7-e43d-4251-b817-79ccf798706d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.172944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a020c7-e43d-4251-b817-79ccf798706d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.173186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a020c7-e43d-4251-b817-79ccf798706d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.175001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 442/898] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:22.180214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.180506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.180782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.200307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:22.201501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:22.274748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 457/899] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:22.427107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a020c7-e43d-4251-b817-79ccf798706d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.432102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:22.546833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a020c7-e43d-4251-b817-79ccf798706d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:22.584654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:22.585290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.585569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.585845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.586431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:22.587825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:22.596798 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:22.597707 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.598622 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 443/900] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:22.603836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.613383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.645180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:22.727610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb4275-76a1-4e23-950b-491c643b249c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:22.734846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:22.739925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aef46d-05d7-4da6-820b-a51598c674c8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:22.766929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:22.767775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.768148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.768515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.769676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:22.772136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:22.785033 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:22.786352 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:22.787273 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:22.788033 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 444/901] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:22.789645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:22.790285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.794862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.794862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.794862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:22.794862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:22.794862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:22.795321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.802000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 458/902] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:22.808283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.913152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b472c0-50b1-487a-b34d-afea1680de6f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:22.944316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:22.952938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afd54e-69c5-484e-8ff4-a7ed7ca2ceb8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:22.960263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:22.968710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:22.969890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.970942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.971642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.972873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:22.973383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:22.975024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:22.982422 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 445/903] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:22.988382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:22.994909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:22.995521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:22.995803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:22.996075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:22.996704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:22.997233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:22.998358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.005010 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 459/904] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:23.010380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.123197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086701c-5092-4e69-b950-ce275906b396 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.128683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d47c-7805-45d6-9b09-c9df7c05413d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.166953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.167580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.167858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.168129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.168850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:23.169153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.169837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.170683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.170916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.171431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.171974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.172967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] RBAC: Adding query filter params (interface=public, service_id=6218d0407fa04bafaf2890ff37ab2cfc, region_id=RegionOne) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:23.173673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.175236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.178562 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 446/905] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:23.183510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.188720 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 460/906] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:23 2022] GET /identity/v3/endpoints?service_id=6218d0407fa04bafaf2890ff37ab2cfc&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:23.193455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.222755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:23.255191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:23.301994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709333e6-6781-418d-a27b-8a90def95b3e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.308576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b30d-4262-4d9e-9e25-634102eece9b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.339799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.340426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.340703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.340973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.341667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] RBAC: Adding query filter params (interface=public, service_id=be1a5a2d7e224492a3a8527184638b3f, region_id=RegionOne) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:23.342028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.342689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.343294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.343389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c7df8-5a72-4143-941a-726943c2e44d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.343571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.343840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.344434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.346473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4375ce8-a806-463f-9701-9e5dcab00b4c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.353563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 447/907] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:23 2022] GET /identity/v3/endpoints?service_id=be1a5a2d7e224492a3a8527184638b3f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:23.353563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18baf07-f263-465d-b6c3-704014612089 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.353563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18baf07-f263-465d-b6c3-704014612089 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.353563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18baf07-f263-465d-b6c3-704014612089 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.353563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 448/908] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:23.355148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18baf07-f263-465d-b6c3-704014612089 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.359479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 461/909] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:23 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:34:23.364920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def31ee-5298-499f-84b0-3ce9cf4d0c3b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.364920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def31ee-5298-499f-84b0-3ce9cf4d0c3b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.364920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def31ee-5298-499f-84b0-3ce9cf4d0c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.381614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0def31ee-5298-499f-84b0-3ce9cf4d0c3b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:23.382829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0def31ee-5298-499f-84b0-3ce9cf4d0c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:23.469749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 462/910] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:23.469749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b8d70-2981-444b-8a52-8ed079590a73 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.469749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b8d70-2981-444b-8a52-8ed079590a73 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.469749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b8d70-2981-444b-8a52-8ed079590a73 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.469749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 463/911] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:23.477685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73a398-603d-401d-9470-155049720ef9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.478052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73a398-603d-401d-9470-155049720ef9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.478403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73a398-603d-401d-9470-155049720ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.482281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18baf07-f263-465d-b6c3-704014612089 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.502059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ac73a398-603d-401d-9470-155049720ef9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:23.503734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac73a398-603d-401d-9470-155049720ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:23.512198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:23.516206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:23.527434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.528156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.528500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.528840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.529640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.532080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.541909 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 449/912] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:23 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:34:23.590566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 464/913] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:23.826485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.935547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73a398-603d-401d-9470-155049720ef9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:23.936318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938c111-a61f-4d9f-885c-338c453e3363 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:23.972140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:23.973131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:23.973620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:23.974081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:23.974768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:23.976137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:23.995102 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 450/914] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:23 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:24.058044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73a398-603d-401d-9470-155049720ef9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.106660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.107567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.107993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.108439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.122044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:24.127824 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:24.128956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:24.129946 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:24.136475 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 465/915] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:24.138869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:24.200750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:24.259356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641db5b0-cb04-4ad6-8eb3-7f60a518dc04 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.296153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.297003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.297467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.297880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.299043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:24.299490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:24.301005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:24.314577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 451/916] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:24.316456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:24.320015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:24.420635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cd74a-0a68-454c-99c4-2885214cf272 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.456182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.457023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.457477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.457904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.468289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:24.469925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:24.485696 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:24.487152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:24.492805 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:24.492805 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:24.492805 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:24.492805 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 466/917] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:24.494471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:24.620053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee89b61-266c-4bd3-8981-003a1b7923a8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.633739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:24.661646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.662257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.662557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.662829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.663473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:24.663766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:24.665140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:24.682670 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 452/918] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:24.684626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:24.744952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:24.792079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed84a3-b653-4cef-98bb-f0b719f11375 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.827468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.828099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.828374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.828695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.837827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:24.839298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:24.850867 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:24.851840 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:24.852762 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:24.856786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 467/919] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:34:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:24.858108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:24.962607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe29983-e1a0-4b06-8ef1-2561c8339287 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:24.997052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:24.997583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:24.997859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:24.998128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:24.998872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:24.999173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.000666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.024825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 453/920] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:25.026262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:25.128978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4936d-df3a-4c87-a6ce-dbe509a2fc8b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:25.143667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:25.164879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:25.165506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.165785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.166054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.177349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.178793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:25.193468 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:25.194366 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:25.195413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:25.195413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:25.195413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:25.195413 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 468/921] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:25.201615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:25.234514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:25.312197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea415b9-59fd-4376-9732-537e7e1dbefc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:25.350535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:25.351533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.352067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.352552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.353397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:25.353889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.355472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.373449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 454/922] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:25.375099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:25.480878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9796ad-cfff-4471-b316-fb7c7acb494b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:25.523610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:25.524253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.524530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.524825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.531903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.533545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:25.545105 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:25.546067 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:25.547274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 469/923] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:25.556831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:25.654992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:25.668565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700b4a1-27ae-4b02-9d88-a17cfb4e8c82 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:25.704014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:25.705227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.705702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.705974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.706749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:25.707048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.708473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0819a-45d9-45d5-9094-8322d98bff87 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.725498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 455/924] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:25.728029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:25.761160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:25.837148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389725-5e1e-4969-9c84-26cf40be0e0e None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:25.872555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:25.873219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.873493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.873764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/f329c05a7437450db2315117edbb65ea/roles/ab8b8d0424e74f6cb1092f08abf63252` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.875975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=f329c05a7437450db2315117edbb65ea, role_id=ab8b8d0424e74f6cb1092f08abf63252)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:25.877619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b20f0-1871-4089-84a6-563f9370014d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:25.894429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 470/925] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:25 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/f329c05a7437450db2315117edbb65ea/roles/ab8b8d0424e74f6cb1092f08abf63252 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:25.982820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b892ca-e83d-4ce1-afbb-cb438c6a2f2c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.983137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b892ca-e83d-4ce1-afbb-cb438c6a2f2c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.983536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b892ca-e83d-4ce1-afbb-cb438c6a2f2c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.985288 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 456/926] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:25.991324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.991624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.991876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.995986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747c38-23ce-44d1-b3cc-bfe7a0cae95e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:25.996273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747c38-23ce-44d1-b3cc-bfe7a0cae95e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:25.996548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747c38-23ce-44d1-b3cc-bfe7a0cae95e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:25.998036 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 457/927] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:26.004990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.005284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.005532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.013335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:26.014582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:26.027918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:26.029266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:26.110526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 458/928] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:26.111255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 471/929] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:26.436144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.450461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.470750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:26.532964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:26.557930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd792-d519-4e18-988a-e246310e9621 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:26.572636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea9545-f8a9-4a6f-ab51-e23cbded6076 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:26.595079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:26.595950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.596232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.596503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.597078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:26.598560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:26.607922 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:26.608902 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:26.609736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 459/930] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:26.615093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:26.615477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.615747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.616032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.616304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.616888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:26.618601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:26.637983 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:26.639039 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 20:34:26.639968 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:26.643432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 472/931] 10.222.0.52 () {56 vars in 1132 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:26.650115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.752594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:26.755178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630c90-62b1-4a3e-8804-2aac1784221c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:26.773613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43aa32-b962-4158-b455-9f01abc0e390 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:26.791903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:26.792508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.792787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.793058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.793682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:26.793984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:26.795450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e2fe3-a5be-4ffe-b111-d7316b8a1c35 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:26.806577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 460/932] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:26.806577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2500-e1c9-48e4-a9a7-82d886ec0bad None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.806577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2500-e1c9-48e4-a9a7-82d886ec0bad None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.806577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d2500-e1c9-48e4-a9a7-82d886ec0bad None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.806577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 461/933] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:26.809554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d2500-e1c9-48e4-a9a7-82d886ec0bad None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.816741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:26.817346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.817623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.817890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.818527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:26.818886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:26.820171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:26.821118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ea15a-38cd-4392-a2ac-0426e2079ddf None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:26.831454 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 473/934] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:26.831454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.831454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.831454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.849421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:26.850631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:26.918184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 474/935] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:26.919048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:26.928856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d2500-e1c9-48e4-a9a7-82d886ec0bad None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:26.974464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:26.975620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:26.976115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:26.976788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:26.978019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:26.978571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:26.980151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:26.992514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 462/936] 10.222.0.52 () {56 vars in 1114 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:26.993881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.015158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:27.051342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41827e3-46dc-439f-95f1-066b3522ef01 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.093675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:27.099443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661764-4481-498c-9192-adf9e701639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.100066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661764-4481-498c-9192-adf9e701639c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.100342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661764-4481-498c-9192-adf9e701639c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.100609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661764-4481-498c-9192-adf9e701639c None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.101236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661764-4481-498c-9192-adf9e701639c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:27.101535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661764-4481-498c-9192-adf9e701639c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.102979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661764-4481-498c-9192-adf9e701639c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.113465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 475/937] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:27.114540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661764-4481-498c-9192-adf9e701639c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.140976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53cb7b-1860-4da1-9432-67116673f787 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.191558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.193234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.194012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.194524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.195417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.197940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.234742 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 463/938] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:27.245061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661764-4481-498c-9192-adf9e701639c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.265055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.293281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.293812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.294093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.294364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.295082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.296519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.302048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:27.322463 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 476/939] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:27.402676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e503d5e-d0d1-47f3-be52-61c9d26b14db None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.434946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:27.463310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.464011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.464355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.465430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.465430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.466718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:27.481353 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:27.482633 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:27.483664 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 464/940] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:27.489588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.604488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72f2dd-51c6-4720-a65b-fc577041a987 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.652297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.652904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.653177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.653442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.654070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:27.654368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.655748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.662407 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 477/941] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:27.668094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.774538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a80cd9-0adc-4267-bb16-6fc622d3a6ef None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.810730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.811340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.811677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.811956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.812588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:27.812890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.814322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.819662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 465/942] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:27.825341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:27.890978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:27.934807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736627-1b51-488a-9d4b-7745b6719bcc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:27.973123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:27.974136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:27.974686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:27.975165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:27.976118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] RBAC: Adding query filter params (interface=public, service_id=4b6778b00b0c44f5880d218c9c1c0e5d, region_id=RegionOne) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:27.976657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:27.978263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77cdde-d11d-4f1b-914e-2e558b8eeb0f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:27.984878 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 478/943] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:27 2022] GET /identity/v3/endpoints?service_id=4b6778b00b0c44f5880d218c9c1c0e5d&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:27.992208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:28.043941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:28.093771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4983-5295-4c06-943a-b7cb5ae95c41 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:28.128963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:28.129570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:28.129841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:28.130107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:28.130730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:28.132096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f48b7-9e78-4a9b-8cac-8c6029ee5e21 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:28.142704 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 466/944] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:27 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:34:28.431466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c45e-66b0-46b5-958d-f6269cb2a5ab None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:28.431656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c45e-66b0-46b5-958d-f6269cb2a5ab None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:28.431928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0c45e-66b0-46b5-958d-f6269cb2a5ab None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:28.433381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 479/945] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:28.438900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:28.439530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:28.440035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:28.459039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:28.460549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:28.534416 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 467/946] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:28.871527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c45e-66b0-46b5-958d-f6269cb2a5ab None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:28.977017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0c45e-66b0-46b5-958d-f6269cb2a5ab None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.002884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:29.016335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.017130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.017603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.017880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.028294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:29.033888 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:29.034990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:29.035825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 480/947] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:29.043049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.090870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:29.148914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea77fe-1fc0-4b1e-b96c-5e1c3cdf0341 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.183566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.184167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.184439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.184706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.185353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:29.185646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:29.187044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:29.195593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 468/948] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:29.203775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.311375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682018d0-4494-43f1-bdc6-a82b516213c6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.346303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.346962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.347233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.347498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.358067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:29.359548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:29.375737 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:29.376894 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:29.377772 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:29.377772 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:29.377772 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:29.377772 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 481/949] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:29.384574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.504299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbff596-ea5b-451e-8de1-cf1cd53250c5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.512174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:29.542558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.543996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.544498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.544971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.545814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:29.546344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:29.548107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:29.560975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 469/950] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:29.567442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.606704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:29.682764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b02f9-71ff-4bb4-adad-a3e8ef3044cb None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.717971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.718627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.718905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.719180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.727414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:29.728959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:29.740011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:29.740932 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:29.741873 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 482/951] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:29.748885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.856543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593876f-18b5-4aa9-97c4-07445bf3adac None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:29.893301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:29.894273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.894807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.895286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.896464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:29.896972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:29.898650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3bea9-fd8f-4e34-b986-f359062b94cc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:29.917581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 470/952] 10.222.0.52 () {56 vars in 1149 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:29.924475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:29.945152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1af-0a3d-42a0-b2b7-4b3445235355 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.945903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1af-0a3d-42a0-b2b7-4b3445235355 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.946384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461a1af-0a3d-42a0-b2b7-4b3445235355 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.948244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 471/953] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:29.953760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75faadc-5d6f-43c4-b7ab-54879311f0e0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:29.954970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75faadc-5d6f-43c4-b7ab-54879311f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:29.955457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75faadc-5d6f-43c4-b7ab-54879311f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:29.974775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b75faadc-5d6f-43c4-b7ab-54879311f0e0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:29.976396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b75faadc-5d6f-43c4-b7ab-54879311f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:29.997250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.029231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.037234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c21034-f4ee-4c30-8995-fe76bbce63a5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.062801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 472/954] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:30.065034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4556-5246-444d-b3ca-df65e96cc11b None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.065319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4556-5246-444d-b3ca-df65e96cc11b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.065619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4556-5246-444d-b3ca-df65e96cc11b None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.071317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 473/955] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:30.073188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.073467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.073735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.080284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.081208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.081698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.081969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.092383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:30.093851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:30.175891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.177640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.178835 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 474/956] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:30.276045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.288783 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 483/957] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:30.296051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.391005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.410255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be7f68-9280-45e4-8f8c-ca6aacd5218b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.413264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.459933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.460476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.460752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.461019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.475127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.476875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:30.492037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:30.493074 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:30.494060 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:30.494060 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:30.494060 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:30.500662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 475/958] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:30.502256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.534807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f16b-154d-43bd-ba5f-24408a96db7b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.576497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.577451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.577936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.578408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.579989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.582099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.586078 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.common [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 20:34:30.606468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.sql.core [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 20:34:30.606468 ncdv-master-1 devstack@keystone.service[99132]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 20:34:30.606468 ncdv-master-1 devstack@keystone.service[99132]: [parameters: {'id': '98b8732c06574f5c9957d55c3f1bbf49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 20:34:30.606468 ncdv-master-1 devstack@keystone.service[99132]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99132) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 20:34:30.608589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafe54b-aad1-4f0a-a56b-738f31b5e39c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98b8732c06574f5c9957d55c3f1bbf49.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98b8732c06574f5c9957d55c3f1bbf49. Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:34:30.627644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result.read() Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 20:34:30.628752 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application yield resource Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise self.value Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application session.commit() Sep 24 20:34:30.630102 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.flush() Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application compat.raise_( Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 20:34:30.631245 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 20:34:30.632338 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result.read() Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:34:30.633369 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [parameters: {'id': '98b8732c06574f5c9957d55c3f1bbf49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.634436 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:30.635410 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 98b8732c06574f5c9957d55c3f1bbf49. Sep 24 20:34:30.636530 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.639721 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 484/959] 10.222.0.52 () {60 vars in 1140 bytes} [Sat Sep 24 20:34:30 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:30.640829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.656368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.657802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.658241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.658687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.659506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:30.659955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.661488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.664608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.675552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 476/960] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:30.676377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.701100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.756120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7598-8045-45b5-a26c-7bda1b596363 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.779695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a2696-23ae-420d-9173-1867c56bb8cd None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.789862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.790490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.790802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.791076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.791649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.793210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:30.802475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:30.803636 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.804489 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 485/961] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:30.805276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.817833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.818461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.818792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.819062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.825719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.827205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-502aef09-23a1-4b6a-b823-e2b1b7242447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:30.838230 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:30.839233 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:30.840137 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 477/962] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:30.840626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638003d-5a98-4672-b0e5-555384c66480 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.840694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638003d-5a98-4672-b0e5-555384c66480 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.840962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638003d-5a98-4672-b0e5-555384c66480 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.842305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 478/963] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:30.842884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638003d-5a98-4672-b0e5-555384c66480 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.902865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.909670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67c4b1-db0a-419f-a425-4786e0f0e837 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.919725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:30.944288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.944937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.945174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.945444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.948288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638003d-5a98-4672-b0e5-555384c66480 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:30.951904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.953340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:30.964734 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:30.965769 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 486/964] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:30.967152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:30.985869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:30.986578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.986912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.987191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:30.987809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] RBAC: Adding query filter params (name=compute) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:30.988107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:30.989476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533a9a9-e4ba-4fe7-9e15-4a96917ea3cb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:30.994003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 479/965] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:30.995147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:30.995427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:30.995701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.013983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:31.015133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:31.070577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596c0fd-ca56-4515-8d7c-5990368710e6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.081234 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 480/966] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:31.082062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.105325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.105926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.106204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.106480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.107155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.107457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.108879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.134381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 487/967] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.135496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.155338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:31.176218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:31.223119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32ed8-a161-49b6-89e8-1b937b6e3067 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.263610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d543be-e14b-44f6-93aa-7ff233a588ee None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.268103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.268661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.268937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.269208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.270095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.270679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.272168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.289876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 481/968] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.299631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.312983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.313923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.314202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.314471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.315139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] RBAC: Adding query filter params (type=compute) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.315443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.316845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.326536 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 488/969] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.419257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.448008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad24796-6dfa-485e-b2c8-2407711b0aa4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.466079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:31.503426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.504052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.504330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.504598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.505325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] RBAC: Adding query filter params (role.id=ab8b8d0424e74f6cb1092f08abf63252, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=f329c05a7437450db2315117edbb65ea) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.505629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.507456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.513771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 482/970] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/role_assignments?user.id=f329c05a7437450db2315117edbb65ea&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=ab8b8d0424e74f6cb1092f08abf63252 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.560811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:31.570467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cfba-9a63-4519-b010-64a542f4f41c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.609519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.610732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.611014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.611283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.611935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.613408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:31.623414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:31.624475 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:31.625297 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:31.635327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 489/971] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:31.642665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.760262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b50e2a-d70e-4851-b847-6b6989b6cd01 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.798453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.799095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.799376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.799659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.800287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.800591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.801968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.807306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 483/972] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.813463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:31.925369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f00c7-2c91-4f00-a5f4-8b980e5d4928 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:31.971231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:31.971881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:31.972127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:31.975866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:31.975866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:31.975866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:31.975866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d590a-8682-442f-9c24-19e21ab2a0c8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:31.980658 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 490/973] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:31.986329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:32.034226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:32.101079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7955f-81af-469c-8c49-1849fac8f10b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:32.137391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:32.138170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:32.138448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:32.138748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:32.139483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:32.140869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269c18-b94a-4ec2-9b89-c4e94e741295 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:32.165061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 484/974] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:33.810489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc858c-81e3-414a-ad0c-0a5021b5dca2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:33.811052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc858c-81e3-414a-ad0c-0a5021b5dca2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:33.811052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc858c-81e3-414a-ad0c-0a5021b5dca2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:33.812647 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 491/975] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:33.817917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08074a70-6fe4-48fc-9e27-c1acc8f6864f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:33.818357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08074a70-6fe4-48fc-9e27-c1acc8f6864f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:33.818451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08074a70-6fe4-48fc-9e27-c1acc8f6864f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:33.838134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-08074a70-6fe4-48fc-9e27-c1acc8f6864f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:33.839398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-08074a70-6fe4-48fc-9e27-c1acc8f6864f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:33.916264 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 485/976] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:33.939742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b145c-1b51-48be-8e25-ea5ade39654d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:33.940017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b145c-1b51-48be-8e25-ea5ade39654d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:33.940294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b145c-1b51-48be-8e25-ea5ade39654d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:33.941897 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 492/977] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:33.950479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305debf-613c-475b-a166-81f07490d4dd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:33.950479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305debf-613c-475b-a166-81f07490d4dd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:33.950479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305debf-613c-475b-a166-81f07490d4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:33.967913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e305debf-613c-475b-a166-81f07490d4dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:33.970081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e305debf-613c-475b-a166-81f07490d4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:33.986344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.043196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 486/978] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:34.096704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8689-51bc-4d9f-8343-4ec17604b40f None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.096983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8689-51bc-4d9f-8343-4ec17604b40f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.097259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8689-51bc-4d9f-8343-4ec17604b40f None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.098828 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 493/979] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:34.104111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.104395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.104671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.123388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:34.124573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:34.196397 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 487/980] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:34.254450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8689-51bc-4d9f-8343-4ec17604b40f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.283874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.371286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8689-51bc-4d9f-8343-4ec17604b40f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.387377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.408888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.409814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.410271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.410757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.411570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:34.413168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:34.430661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.sql.core [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 20:34:34.430661 ncdv-master-1 devstack@keystone.service[99132]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 20:34:34.430661 ncdv-master-1 devstack@keystone.service[99132]: [parameters: {'user_id': '9d8289c2620e47fc8179faebaf983ed1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 20:34:34.430661 ncdv-master-1 devstack@keystone.service[99132]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99132) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result.read() Sep 24 20:34:34.440208 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:34:34.441140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application yield resource Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise self.value Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application session.commit() Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 20:34:34.442092 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.flush() Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application compat.raise_( Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 20:34:34.443013 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 20:34:34.443898 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application conn.query(q) Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result.read() Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 20:34:34.444736 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [parameters: {'user_id': '9d8289c2620e47fc8179faebaf983ed1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:34.445614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 24 20:34:34.447817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.450295 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 494/981] 10.222.0.52 () {60 vars in 1131 bytes} [Sat Sep 24 20:34:34 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:34.452439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.507986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.533415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71dad0-3182-49f6-ac7f-fe477d5ed98d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.556315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.574058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f332bd-27ae-48eb-bda9-e19b8e181e78 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.577413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.578021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.578301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.579147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.579807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:34.581382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:34.597058 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 488/982] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:34 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:34.597861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.624595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.625769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.626264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.626872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.639089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:34.641048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:34.658138 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:34.659269 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 20:34:34.660255 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:34.664158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 495/983] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:34.665585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.751988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7de2f9-fea7-4048-9921-4f4dd10f895e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.790915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.791470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.791745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.792011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] PATH_INFO: `/v3/services/image` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.792601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:34.794013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:34.800450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50482166-32f0-40f8-b900-0fb990ce2e11 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-e25b16f2-994f-4319-8f63-aae840875261 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:34.803925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:34.804932 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:34.811944 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 489/984] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 20:34:34 2022] GET /identity/v3/services/image => generated 86 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:34.812748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec7ec-dd94-4211-9f71-8172cf9edc0d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.812748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec7ec-dd94-4211-9f71-8172cf9edc0d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.812748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ec7ec-dd94-4211-9f71-8172cf9edc0d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.812748 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 490/985] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:34.812748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ec7ec-dd94-4211-9f71-8172cf9edc0d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:34.816507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.847606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.848217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.848496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.848881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.849463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:34.849767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:34.851252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9c8fb-4ef8-4bab-88f6-3cae25205ac3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:34.875488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:34.894471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 496/986] 10.222.0.52 () {56 vars in 1101 bytes} [Sat Sep 24 20:34:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:34.894471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.894471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.894471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.911704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:34.913698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:34.946948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ec7ec-dd94-4211-9f71-8172cf9edc0d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:34.994636 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 497/987] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:34.996903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:34.997745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:34.998239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:34.998737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:34.999558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] RBAC: Adding query filter params (name=image) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:35.000035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.002238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.008363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 491/988] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:35.013739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:35.123322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ad295-fb2a-4347-8b47-8f149cfeafdc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:35.150867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:35.159580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:35.160192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:35.160474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:35.160749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:35.161383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:35.161689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.163101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.168232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 498/989] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:35.329956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:35.406118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:35.441595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786495d-8f9e-41e5-8ddb-796415417b49 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:35.479601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:35.480228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:35.480505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:35.480777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:35.481354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.482758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:35.491800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:35.492752 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:35.493574 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:35.494980 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 492/990] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:35.499120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:35.601176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e7bc-4258-429c-b4c5-8f61562098d2 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:35.636338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:35.637415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:35.637922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:35.638426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:35.639070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:35.639367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.640726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.650581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 499/991] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:35.652118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:35.755480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3773f-d6b0-448a-804d-9ffaeb7d3ecc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:35.788477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:35.795171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:35.795772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:35.796041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:35.796305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:35.796922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:35.797217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.799276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.804257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 493/992] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:35.809981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:35.868994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:35.915586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4856a9-50a6-48dd-93e3-f83e001cc8e9 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:35.948484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:35.949094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:35.949375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:35.949645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:35.950348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] RBAC: Adding query filter params (interface=admin, service_id=4b6778b00b0c44f5880d218c9c1c0e5d, region_id=RegionOne) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:35.950697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:35.952062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23e4a0-6a4b-4972-b90a-d73035accc98 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:35.962367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 500/993] 10.222.0.52 () {56 vars in 1249 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/endpoints?service_id=4b6778b00b0c44f5880d218c9c1c0e5d&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:35.964256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:36.067591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c8426-f28d-4e0d-ba9c-cf1dcfb1b83d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:36.101607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:36.102212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:36.102490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:36.102798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:36.103397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:36.104770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee091-2830-4648-86fc-59e0413c5497 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:36.118542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 494/994] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:35 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:34:37.114959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdf12-91ae-4c81-9940-71602abe8cce None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.115581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdf12-91ae-4c81-9940-71602abe8cce None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.115581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcdf12-91ae-4c81-9940-71602abe8cce None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.116978 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 501/995] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:37.121991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955297-09bf-46c2-a81f-0313b69b577a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.122402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955297-09bf-46c2-a81f-0313b69b577a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.122638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7955297-09bf-46c2-a81f-0313b69b577a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.141603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f7955297-09bf-46c2-a81f-0313b69b577a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:37.142815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f7955297-09bf-46c2-a81f-0313b69b577a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:37.205969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:37.218656 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 495/996] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:37.549159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcdf12-91ae-4c81-9940-71602abe8cce None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:37.606747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501994b1-4565-447d-b0cd-7498d0e804da None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.607073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501994b1-4565-447d-b0cd-7498d0e804da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.607289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501994b1-4565-447d-b0cd-7498d0e804da None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.608607 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 496/997] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:37.613809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.614087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.614358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.635378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:37.635378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:37.661120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcdf12-91ae-4c81-9940-71602abe8cce None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:37.682884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:37.698224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:37.698887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.699167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.699439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.700184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:37.701882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:37.708663 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 497/998] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:37.711251 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:37.712294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:37.713244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 502/999] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:37.718533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:37.820550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921373d-1d4a-4541-8c43-e62b41a3a9d5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:37.840698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:37.855957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:37.856558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:37.856835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:37.857107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:37.857723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] RBAC: Adding query filter params (name=compute) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:37.858024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:37.859437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:37.864375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 498/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:37.869838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:37.972978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446c4b-ddb2-458e-9c74-19bfb669fcd6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:38.008521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:38.009118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.009403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.009660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.010275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] RBAC: Adding query filter params (type=compute) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:38.010609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:38.011968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:38.022370 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 503/1001] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:38.023349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:38.043121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:38.095367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:38.131954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed81bf-0ac6-4fea-b724-dd4005be663f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:38.153471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810056c-7188-4d5e-b51f-afc536519523 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:38.170791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:38.171638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.171922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.172304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.173084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] RBAC: Adding query filter params (interface=public, service_id=75e55472e3134f2293028012fae10b98, region_id=RegionOne) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:38.173389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:38.174790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:38.180065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 499/1002] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:38 2022] GET /identity/v3/endpoints?service_id=75e55472e3134f2293028012fae10b98&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:38.185371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:38.191157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:38.191769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.192049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.192322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.192967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:38.194340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5181c88-f943-427a-9259-e2db2f2ae3cc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:38.220164 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 504/1003] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:38 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:34:38.244327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:38.305250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f380d9-9b81-4569-a4a0-6481f1c71a0f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:38.349875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:38.350403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.350717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.350987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.351588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:38.352976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bac55-731f-4e5e-9beb-c05028df1906 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:38.360353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 500/1004] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:38 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:34:38.573217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180af1e-a3d6-4c84-91b1-b6b24caa1cce None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.573509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180af1e-a3d6-4c84-91b1-b6b24caa1cce None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.573777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180af1e-a3d6-4c84-91b1-b6b24caa1cce None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.575136 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 505/1005] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:38.581226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:38.581496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:38.581767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:38.600173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:38.601262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:38.674390 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 501/1006] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:39.009212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180af1e-a3d6-4c84-91b1-b6b24caa1cce None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:39.116986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180af1e-a3d6-4c84-91b1-b6b24caa1cce None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:39.126550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:39.153077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:39.153700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:39.154252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:39.154252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:39.154817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:39.156175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:39.165857 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:39.166913 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:39.167783 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 506/1007] 10.222.0.52 () {56 vars in 1126 bytes} [Sat Sep 24 20:34:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:39.174681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:39.215241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:39.277937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62370808-b1e9-44a7-a99d-6480062e9f58 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:39.312212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:39.313354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:39.313628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:39.313898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:39.314539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:39.314843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:39.316246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:39.325778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 502/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:39.327510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:39.432163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef6c7f-12ba-4970-915c-cbeeeaf36b4d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:39.467649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:39.468269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:39.468544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:39.468810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:39.470089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:39.470390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:39.471982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d08dd1-f472-4bd5-9954-758c8541c1a2 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:39.482307 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 507/1009] 10.222.0.52 () {56 vars in 1122 bytes} [Sat Sep 24 20:34:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:39.483607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:39.590016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346d080-1d81-48b2-b00e-927af9b07a37 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:39.591739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:39.627130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:39.627740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:39.628017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:39.628290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:39.631904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:39.631904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c05e7-6662-408d-981d-6d6dfd406b5c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:39.658527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 503/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:40.682551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34958b-4d83-4a61-b1a9-ee7d4df1991c None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:40.683085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34958b-4d83-4a61-b1a9-ee7d4df1991c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:40.683085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34958b-4d83-4a61-b1a9-ee7d4df1991c None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:40.684460 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 508/1011] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:40.689623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a84f0-9be7-4199-aae4-22acdeb10f36 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:40.690011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a84f0-9be7-4199-aae4-22acdeb10f36 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:40.690184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a84f0-9be7-4199-aae4-22acdeb10f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:40.708819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8e6a84f0-9be7-4199-aae4-22acdeb10f36 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:40.709999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e6a84f0-9be7-4199-aae4-22acdeb10f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:40.787329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 504/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:40.833592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f79e8-e986-454d-a7c1-77f2be72c72f None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:40.833915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f79e8-e986-454d-a7c1-77f2be72c72f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:40.834148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f79e8-e986-454d-a7c1-77f2be72c72f None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:40.835654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 509/1013] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:40.840715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:40.841010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:40.841288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:40.860874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:40.862059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:40.929927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 505/1014] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:40 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:41.116678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f79e8-e986-454d-a7c1-77f2be72c72f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.152686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.217244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f79e8-e986-454d-a7c1-77f2be72c72f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.250288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.250941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.251216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.251482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] PATH_INFO: `/v3/services/image` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.252049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.253474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.253867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:41.262308 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:41.263512 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:41.264620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:41.269330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 510/1015] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:41.270476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.280624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.373930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3050b9-8ebc-47c7-bb07-ebfb8f91064a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.382006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.385195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88211c9e-ab71-4cfb-a986-a2c0d9503574 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.411509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.412137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.412410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.412679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.413271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.414665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:41.424792 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:41.425795 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:41.426660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 506/1016] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:41.431648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.432546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.433150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.433425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.433714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.434331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] RBAC: Adding query filter params (name=image) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:41.434671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.436024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.446896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 511/1017] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:41.447801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.529643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.550272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779669a-9c5f-4328-87dd-53996164f9b6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.558723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834bbe-ff9d-45ca-9e22-2c1af5ecefd3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.586857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.587777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.588248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.588759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.589689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:41.590181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.591785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.597709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.598312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.598841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.599305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.600417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] RBAC: Adding query filter params (type=image) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:41.600951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.602737 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 507/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:41.602862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.603404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.607946 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 512/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:41.613916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.652387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.728414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcec200-6a11-4982-bd4a-b5f809e87ca1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.734130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05f28e-4f20-4e05-a0d2-4da19dfce0ad None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.765297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.766931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.767173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.767625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.768727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] RBAC: Adding query filter params (type=compute) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:41.769226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.771995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.772116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.772599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.772875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.773144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.773845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] RBAC: Adding query filter params (interface=public, service_id=e08e48b5c0c646a6893abd2bb7ce2862, region_id=RegionOne) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:41.774143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.775526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.780743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 508/1020] 10.222.0.52 () {56 vars in 1112 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:41.785792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 513/1021] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/endpoints?service_id=e08e48b5c0c646a6893abd2bb7ce2862&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:41.787246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.787392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:41.801664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.885970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:41.897774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42149f57-b2f3-473c-bca4-d3f2dc8c550a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.898869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cfac7-9ad2-4e05-8e27-47e1ecd4d83e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:41.932733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.933324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.933604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.933939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.934560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.935933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb278d-f3b8-4f96-bb78-3d9b90853e54 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.936927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:41.937595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:41.937866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:41.938138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:41.938900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:41.940245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7358732-cbe2-44e5-8ddd-28bb87dd9a6b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:41.946881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 514/1022] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:34:41 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:34:41.969501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 509/1023] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:42.133001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fe334-2f52-4f84-888d-a3e43a0b26e0 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:42.133362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fe334-2f52-4f84-888d-a3e43a0b26e0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:42.133559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fe334-2f52-4f84-888d-a3e43a0b26e0 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:42.135225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 515/1024] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:42.141579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:42.141887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:42.142147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:42.163582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:42.164805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:42.231873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:42.248208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 510/1025] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:42.575958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fe334-2f52-4f84-888d-a3e43a0b26e0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:42.684341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fe334-2f52-4f84-888d-a3e43a0b26e0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:42.720344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:42.720999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:42.721278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:42.721548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:42.722691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:42.724110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b5f11-748f-4854-b5b8-cdde3f61bc7a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:42.725856 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 516/1026] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:42.731444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:42.837942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740618c-d345-4e4f-a9d1-e3fc6e0dacb3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:42.872422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:42.873029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:42.873309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:42.873580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:42.874226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:42.875732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17144b17-dc48-49e4-b52a-46a7fd6a88bf None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:42.910523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 511/1027] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:34:42 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:44.429769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bea0-c74e-4fdd-93f6-6ff53a735163 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:44.430303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bea0-c74e-4fdd-93f6-6ff53a735163 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:44.430303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1bea0-c74e-4fdd-93f6-6ff53a735163 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:44.431828 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 517/1028] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:44.436839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75d28-23f1-4c1a-ac99-51a3edabe4e2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:44.437318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75d28-23f1-4c1a-ac99-51a3edabe4e2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:44.437398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75d28-23f1-4c1a-ac99-51a3edabe4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:44.461289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b4b75d28-23f1-4c1a-ac99-51a3edabe4e2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:44.462475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b4b75d28-23f1-4c1a-ac99-51a3edabe4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:44.502711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:44.532890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 512/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:44.543956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98bbb7-d36e-4759-8428-5b676a6a3316 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:44.544246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98bbb7-d36e-4759-8428-5b676a6a3316 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:44.544520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98bbb7-d36e-4759-8428-5b676a6a3316 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:44.546562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 518/1030] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:44.551252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:44.551542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:44.551818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:44.570523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:44.571784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:44.647088 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 513/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:44.865932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98bbb7-d36e-4759-8428-5b676a6a3316 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:44.892235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:44.976404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98bbb7-d36e-4759-8428-5b676a6a3316 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:44.983757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.015201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.017225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.017515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.017783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] PATH_INFO: `/v3/services/image` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.018353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.020217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:45.029431 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:45.030465 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:45.031646 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 519/1032] 10.222.0.52 () {56 vars in 1105 bytes} [Sat Sep 24 20:34:44 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:45.036753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.093306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84b193-c06a-4e47-bf8e-ffd01487bd34 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.095782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.131369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.131918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.132194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.132464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.133296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.145115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:45.147312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ca292-b1a7-4896-afa0-f92fb6ad6991 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:45.149910 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:45.150989 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:45.151832 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 514/1033] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:45.158560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.186114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.186744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.187031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.187453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.188321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] RBAC: Adding query filter params (name=image) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:45.188577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.189878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.196125 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 520/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:45.201518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.271690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cbc44-ae39-4349-9bb0-a7eabbd2fc00 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.308858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bccea-04be-434c-bc7e-967281864263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.309453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bccea-04be-434c-bc7e-967281864263 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.309726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bccea-04be-434c-bc7e-967281864263 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.309992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bccea-04be-434c-bc7e-967281864263 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.310323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30bcf-3309-40ba-8538-534d80a20a56 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.310675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bccea-04be-434c-bc7e-967281864263 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:45.310974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bccea-04be-434c-bc7e-967281864263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.312913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bccea-04be-434c-bc7e-967281864263 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.322592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 515/1035] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:45.327691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bccea-04be-434c-bc7e-967281864263 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.347951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.348560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.348839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.349170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.349795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] RBAC: Adding query filter params (type=image) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:45.350150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.351603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed195e4-68cb-4602-9298-113fd7373b6d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.356885 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 521/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:45.357114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.358321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1249-dbbc-4678-9e1c-d4eeccb3ba3d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.358462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1249-dbbc-4678-9e1c-d4eeccb3ba3d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.359761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b1249-dbbc-4678-9e1c-d4eeccb3ba3d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.361213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 522/1037] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:45.366024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b1249-dbbc-4678-9e1c-d4eeccb3ba3d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.385401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.433692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bccea-04be-434c-bc7e-967281864263 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.468331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.468968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.469244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.469355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b1249-dbbc-4678-9e1c-d4eeccb3ba3d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.469740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.479919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.481379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5a34381b-72c6-4f26-9f49-0580cfe1a8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:45.495226 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:45.496303 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:45.497216 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:45.497216 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:45.497216 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:45.497216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 516/1038] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:45.497658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.497726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.497990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.507848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.508465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.508720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.508983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.509632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.511015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.517209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:45.518393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:45.542523 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 523/1039] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:34:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:45.542523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.605203 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 517/1040] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:45.628735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.674910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ebaf3-06ee-4ae2-b2ce-97aa99888b6b None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.712756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.713433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.713828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.714121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.714804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:45.715108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.716497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.735376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 524/1041] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:45.736906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.740531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:45.842369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2af868-91fc-4aa8-aa39-b7c731cf2adb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:45.879074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:45.879978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:45.880405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:45.880823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:45.888170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:45.889776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:45.901944 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:45.903023 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:45.903971 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:45.904530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 518/1042] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:45.910474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:45.942527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.005967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.020784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d59c1-3dcf-4a7b-9038-1afd067333e4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.048712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3884db0-0252-42da-b69d-ab09b23f1945 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.052849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.059468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.060076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.060421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.060623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.061346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:46.061812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.064025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.087850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 525/1043] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:46.090814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.091714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.092150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.092434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.092704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.094648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.097015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.103180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 519/1044] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:46.104080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.199303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6294ee9-6135-4781-b3ac-2cbe8b1c7660 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.224822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa6cde-a7c7-4c7e-a9e0-e40d6106425f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.237554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.238555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.238938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.239214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.254694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.260631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.261240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.261523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.261798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.262444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.263922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.288533 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 520/1045] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:34:46 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:34:46.322241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.336103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.337618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.462370 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 526/1046] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 20:34:46.463787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.575113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09e5b1-57a9-4d44-8367-c520a65f14ef None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.610238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.610909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.611185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.611452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.621947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.623417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:46.639048 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:46.639986 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:46.641739 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:46.641739 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:46.641739 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:46.641739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 521/1047] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:46.646848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.742533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.754870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e2da5-f537-4856-975b-2c8bcea2c4ea None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.790478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.791504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.792005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.792490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.793323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:46.793837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.795428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.807960 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 527/1048] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:46.813758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:46.860162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:46.924323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0381-e665-471f-9872-ca3fdd8491d4 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:46.959375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:46.960017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:46.960284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:46.960558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:46.967221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:46.968699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:46.979880 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:46.980810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:46.981715 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 522/1049] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:46.989411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:47.089062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5557a-51fa-4609-a462-8c479d19c5ad None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:47.122663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:47.123266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:47.123540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:47.123807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:47.124521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:47.124913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:47.126862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160be89-2106-4037-be77-97ae90385f32 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:47.143478 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 528/1050] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:47.145364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:47.243083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:47.246969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65f547-1dc5-471e-854e-e5de11fb50b0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:47.281299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:47.281996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:47.282272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:47.282567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:47.283280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] RBAC: Adding query filter params (role.id=ab8b8d0424e74f6cb1092f08abf63252, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:47.283584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:47.285013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980cda0-d1f0-4a04-b890-d0dc88592141 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:47.298579 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 523/1051] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:47 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=ab8b8d0424e74f6cb1092f08abf63252 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:48.030902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f66c4-8adf-4570-bea6-1b73dfa7a309 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.031476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f66c4-8adf-4570-bea6-1b73dfa7a309 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.031476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f66c4-8adf-4570-bea6-1b73dfa7a309 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.032884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 529/1052] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:48.037956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.038375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.038569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.058092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:48.059317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:48.127874 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 524/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:48.456877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f66c4-8adf-4570-bea6-1b73dfa7a309 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:48.470467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:48.562511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f66c4-8adf-4570-bea6-1b73dfa7a309 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:48.596962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:48.597566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.597841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.598108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.674458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:48.675937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53337bab-17ac-4de0-8d76-4fdb29b57667 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:48.688315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:48.782296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 530/1054] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:48.787833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:48.791499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931f97-cda8-4261-95af-c16d0bb41126 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.791784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931f97-cda8-4261-95af-c16d0bb41126 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.792057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931f97-cda8-4261-95af-c16d0bb41126 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.793482 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 531/1055] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:48.798582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.798872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.799149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.817553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:48.818786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:48.898537 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 532/1056] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:48.899008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:48.907173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed6631-8cb0-4ae1-93c2-203ca553069d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:48.942128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:48.942804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:48.943086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:48.943358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:48.954129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:48.955623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:48.971399 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:48.972345 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:48.978596 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:48.978596 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:48.978596 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:48.978596 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 525/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:48.979554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:49.021863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:49.080297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b5357-5ca6-4fb9-bd32-719cd9465818 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:49.113925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:49.114544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.114822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.115089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.115714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:49.116009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:49.117349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:49.134534 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 533/1058] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:49.239311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:49.360527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fa824-0158-4796-a41d-d50fbbcba43b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:49.397316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:49.398190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.398588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.398885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.415997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:49.420818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:49.422307 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:49.423617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 526/1059] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:49.428975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:49.532734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:49.536026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118688f1-06da-43a7-af4e-d8e4868cc78c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:49.571254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:49.571851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.572132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.572406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.573055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:49.573358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:49.574793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3a99-ebee-42a7-9b61-d7933046ac85 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:49.583672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 534/1060] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:49.589727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:49.598931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:49.692301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea466378-3780-4fbd-b659-5d7925211aa7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:49.725328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:49.725950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.726227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.726519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] PATH_INFO: `/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.740365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] RBAC: Authorizing `identity:get_user(user_id=a28c706911b2406dbc5abd2e5ff1d7a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:49.741815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:49.743481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 527/1061] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:34:49 2022] GET /identity/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:49.743745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4ff9-7486-4aeb-ba97-0d64a3bf2aec None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.744035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4ff9-7486-4aeb-ba97-0d64a3bf2aec None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.744309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4ff9-7486-4aeb-ba97-0d64a3bf2aec None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.745711 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 535/1062] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:49.748836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:49.750925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.751212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.751490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.772998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:49.774590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:49.813928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:49.844605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 536/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:49.853174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757686-368f-42a3-91fe-447a737980fe None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:49.886365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:49.886931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:49.887204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:49.887470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:49.952273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:49.954047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.011326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.049557 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 528/1064] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:34:50.055433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.143186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.152304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140f7e1-d50e-41e2-9ccb-329a67f8232e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.175941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.185549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.186176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.186451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.186777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.187886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.189302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.190952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 537/1065] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:50.196810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.277984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921cbb-df2d-4868-b61a-f160522e904c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.295743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09118be1-3a9d-460f-bf3b-43617f591712 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.312324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.312961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.313239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.313511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.323339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:50.328371 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:50.329503 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:50.330708 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 529/1066] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:50.332545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.333140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.333413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.333676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.334444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=03fdfc67d1064bc899556d06d686be04, user.id=a28c706911b2406dbc5abd2e5ff1d7a4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:50.334708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.335413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.336104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.341722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 538/1067] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/role_assignments?user.id=a28c706911b2406dbc5abd2e5ff1d7a4&scope.project.id=03fdfc67d1064bc899556d06d686be04&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:50.362565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.448928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90bf63-bdff-47a6-b77c-cc7c29428b81 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.484926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.485547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.485821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.486090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.486800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:50.487100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.488543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.497547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 530/1068] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:50.504599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.532126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.606696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5047-8063-48b3-8746-a1dffb089489 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.645170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.645798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.646077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.646350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.656861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.658294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:50.672179 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:50.673287 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:50.674189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:50.674189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:50.674189 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:50.674189 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 539/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:50.679075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.762099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.777245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a6b9-a740-407b-a381-618edf260901 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.810031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.810659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.810939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.811216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.812515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:50.812871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.814238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.825617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 531/1070] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:50.831562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:50.908271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:50.931414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a309666-ef00-40a2-b653-c90099620db3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:50.964399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:50.965018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:50.965298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:50.965569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:50.972902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:50.974321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:50.985781 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:50.986880 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:50.987817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 540/1071] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:50.992847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.090230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c326b-b601-4b84-88d4-1fbfeb6eaadf None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.123223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.123820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.124094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.124957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.125673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:51.125972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.127428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.144116 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 532/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:34:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:51.149968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.247889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d3719-cc20-4bf7-89c3-e89a30ad5124 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.279959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.281059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.281328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.281595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.284157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:51.292753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.294185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:51.308495 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:51.309480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:51.310367 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:51.310367 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:51.310367 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:51.310367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 541/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:51.315718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.319474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:51.424837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669ada2-ef9d-4014-b089-c3bbe2726277 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.458319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.458933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.459208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.459472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.460100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:51.460397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.461757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.475632 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 533/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:51.481125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.579353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d079d16-9d16-46f6-80d4-01be64c9de46 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.612725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.613349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.613623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.613888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.620378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.621807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:51.632956 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:51.633889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:51.634828 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 542/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:51.640564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.702106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:51.740382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97cf10-41b1-4304-a098-bdd327423260 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.773877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.774486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.774787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.775060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.775746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:51.776052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.777484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97240bb-52be-48cf-a17d-87112932170d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.791525 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 534/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:51.797369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:51.845824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:51.894195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668521d-8e91-4de6-9d1c-dfa22da58e55 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:51.926564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:51.927200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:51.927484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:51.927754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/ab8b8d0424e74f6cb1092f08abf63252` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:51.929865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=6531dafbedef45ccb7b6b7e2f610c2c2, role_id=ab8b8d0424e74f6cb1092f08abf63252)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:51.931322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87225757-1429-4ea0-9572-3a9b099f0b36 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:51.943681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 543/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:51 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/6531dafbedef45ccb7b6b7e2f610c2c2/roles/ab8b8d0424e74f6cb1092f08abf63252 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:52.730359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ab0-e67e-47f2-abb4-ac81fa34d908 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:52.730910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ab0-e67e-47f2-abb4-ac81fa34d908 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:52.730910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3ab0-e67e-47f2-abb4-ac81fa34d908 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:52.732496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 535/1078] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:52.737454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:52.737853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:52.737998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:52.756130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:52.757286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:52.820659 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 544/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:53.146094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3ab0-e67e-47f2-abb4-ac81fa34d908 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:53.245413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3ab0-e67e-47f2-abb4-ac81fa34d908 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:53.268803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:53.280054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:53.281236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:53.281516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:53.281784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:53.291593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:53.296363 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:53.297514 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:53.298693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 536/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:53.303453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:53.341555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:53.399994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1a1e6-d75a-4990-9467-b8530c6aa161 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:53.432616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:53.433205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:53.433478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:53.433742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:53.434382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:53.434706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:53.436058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:53.444053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 545/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:53.449668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:53.548951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb38a-0798-44ae-afec-029ab1be843a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:53.583066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:53.583688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:53.583965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:53.584234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] PATH_INFO: `/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:53.585366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=a28c706911b2406dbc5abd2e5ff1d7a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:53.586852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:53.588418 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 537/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:34:53 2022] GET /identity/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:53.594361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:53.690314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:53.693252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd702af-b7cb-495b-8bb3-1e842e7e3f92 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:53.727629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:53.728250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:53.728528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:53.728798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:53.729893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:53.731337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af722e48-d420-4228-b550-bef7ac118cac None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:53.733012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 546/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:34:53 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:53.738956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:53.798304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:53.844968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee2db2-1c4c-4b8e-bb03-73794a054ab6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:53.878719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:53.879367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:53.879641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:53.879911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04/users/a28c706911b2406dbc5abd2e5ff1d7a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:53.881768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] RBAC: Authorizing `identity:create_grant(project_id=03fdfc67d1064bc899556d06d686be04, user_id=a28c706911b2406dbc5abd2e5ff1d7a4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:53.883222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4818a-b5a1-4363-bae5-12e8f11b9591 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:53.896407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 538/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:34:53 2022] PUT /identity/v3/projects/03fdfc67d1064bc899556d06d686be04/users/a28c706911b2406dbc5abd2e5ff1d7a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:34:54.341509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92445b-dcd9-4da6-8134-ba5656125e65 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:54.342042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92445b-dcd9-4da6-8134-ba5656125e65 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:54.342042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92445b-dcd9-4da6-8134-ba5656125e65 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:54.344002 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 547/1085] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:34:54.349258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:54.349544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:54.349821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:54.368312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:54.369501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:54.436434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 539/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:34:54.759860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92445b-dcd9-4da6-8134-ba5656125e65 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:54.858648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92445b-dcd9-4da6-8134-ba5656125e65 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:54.891596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:54.892148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:54.892424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:54.892693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:54.902614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:54.907858 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:54.908760 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:54.909594 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 548/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Sat Sep 24 20:34:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:54.915302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:54.958778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:55.015305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed762913-dd32-43a0-b340-7997fe4b969e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.048624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c7178-8732-4975-9b73-6423e501885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.049233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c7178-8732-4975-9b73-6423e501885a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.049539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c7178-8732-4975-9b73-6423e501885a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.049812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c7178-8732-4975-9b73-6423e501885a None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.050478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c7178-8732-4975-9b73-6423e501885a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:55.050808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c7178-8732-4975-9b73-6423e501885a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.052190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c7178-8732-4975-9b73-6423e501885a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:55.060308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 540/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:34:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:55.066702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:55.094859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:55.165163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a093212-e6ba-4ea5-8741-47fb8f387131 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.197835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.198457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.198762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.199029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.208810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.210439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:55.225881 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:55.227140 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:55.228014 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:55.228014 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:55.228014 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:55.228014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 549/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:55.232807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c7178-8732-4975-9b73-6423e501885a None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:55.333717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c7178-8732-4975-9b73-6423e501885a None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.336025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:55.368888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.369523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.369856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.370240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.370960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:55.371263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.372628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:55.384526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 541/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:55.391092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:55.473811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:55.492483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff191f-1c29-4dbf-a426-b70486428ff6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.524754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.525401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.525678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.525943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] PATH_INFO: `/v3/users/nova` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.533871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.535317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:34:55.546113 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:55.547229 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:55.548187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 550/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Sat Sep 24 20:34:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:34:55.552942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:55.651075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a68776-7298-497d-bd44-c4e5c96a482e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.684882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.685475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.685747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.686011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.687021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:55.687331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.688761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:55.706373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 542/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Sat Sep 24 20:34:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:55.712221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:55.810419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421508f-1e22-47af-8d69-19f4ba4f6648 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:55.844053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:55.844653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:55.844934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:55.845201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:55.856432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:55.915738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:55.917176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.013057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 551/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 20:34:56.018977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.043286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.118412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf6870-d24d-429b-ab5e-0da392fed296 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.152621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.153265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.153539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.153806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.165119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:56.167274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-90d3fae4-10e7-46f5-a8e1-e96ac4645cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:34:56.184293 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:56.185452 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:34:56.191129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:34:56.191129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:34:56.191129 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:56.191129 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 543/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:56.191706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.234599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.291990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3a0b0-c465-41c9-b235-2651b94bec66 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.294902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcf74-7aee-4ef9-8e3a-9830e6c770f5 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.295185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcf74-7aee-4ef9-8e3a-9830e6c770f5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.295456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcf74-7aee-4ef9-8e3a-9830e6c770f5 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.297214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 544/1095] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:34:56.302268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.302574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.302865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.320655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:34:56.321836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:34:56.325544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.326140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.326418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.326717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.328004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:56.328294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:56.329637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.340680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 552/1096] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:56.347065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.389214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 545/1097] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:34:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:34:56.442682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.446379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e29c8-6faa-4860-b89e-63939c2d7aa4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.479131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.479748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.480024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.480293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.487278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:56.488696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:34:56.499697 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:56.500744 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:34:56.501637 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 553/1098] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:56.506658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.510628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.604893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb767f1-a49d-4472-b112-0fcea9453d79 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.641353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.641892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.642163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.642427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.643169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:56.643468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:56.645158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.657057 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 546/1099] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:56.663013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.722798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.759499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab6c44-2ba3-497e-bb3e-3097e5abdaf8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.784507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.794279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.794911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.795191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.795464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.796748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] RBAC: Adding query filter params (role.id=ab8b8d0424e74f6cb1092f08abf63252, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=6531dafbedef45ccb7b6b7e2f610c2c2) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:56.797057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:56.798467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:56.804436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 554/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/role_assignments?user.id=6531dafbedef45ccb7b6b7e2f610c2c2&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=ab8b8d0424e74f6cb1092f08abf63252 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:34:56.825288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f342834-ba00-4514-9aaa-136da9fccb73 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:56.863185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:56.863830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:56.864112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:56.864385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:56.876229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:34:56.881698 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:34:56.882835 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:34:56.883667 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 547/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:34:56.889256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:56.917129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:56.994663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379766e3-b85f-4f60-8d15-88b256451c1d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:57.029791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:57.031032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:57.031399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:57.031676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:57.032328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:57.032625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:57.033984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:57.042695 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 555/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:57.049352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:57.152802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa45f3d-c5be-417b-b23e-3ea2e598b701 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:57.177281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:57.189006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:57.189635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:57.189912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:57.190183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] PATH_INFO: `/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:57.191339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] RBAC: Authorizing `identity:get_user(user_id=a28c706911b2406dbc5abd2e5ff1d7a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:57.192780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:57.194353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 548/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:34:57 2022] GET /identity/v3/users/a28c706911b2406dbc5abd2e5ff1d7a4 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:57.200342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:57.289106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:57.304540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b000a4f-cac9-4b32-a852-1ec621224448 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:57.341286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:57.341887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:57.342157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:57.342424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:57.411157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:57.412612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:57.497874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:34:57.515714 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 556/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:34:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 20:34:57.523302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:57.627683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb528e-b815-4a15-b431-8898d52e057f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:57.661987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:57.662657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:57.662939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:57.663213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:57.664337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:57.665772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4815898-96ee-41b0-b8b2-19f80821c80b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:57.667451 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 549/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:34:57 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:34:57.674638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:34:57.775070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad256ba4-0c03-45cb-888c-932b0c1d53ab None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:34:57.808092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:34:57.808682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:34:57.808961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:34:57.809223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:34:57.809923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=03fdfc67d1064bc899556d06d686be04, user.id=a28c706911b2406dbc5abd2e5ff1d7a4) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:34:57.810224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:34:57.812559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fa11d-2d46-4d27-b9fb-794a37f9a79d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:34:57.818080 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 557/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:34:57 2022] GET /identity/v3/role_assignments?user.id=a28c706911b2406dbc5abd2e5ff1d7a4&scope.project.id=03fdfc67d1064bc899556d06d686be04&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:00.243507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f9d6-a7ee-4fda-82a3-367dcb3e7914 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:00.244056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f9d6-a7ee-4fda-82a3-367dcb3e7914 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:00.244056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539f9d6-a7ee-4fda-82a3-367dcb3e7914 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:00.245363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 550/1107] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:00.251188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:00.251617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:00.251735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:00.269685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:00.270893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:00.314760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:00.334813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 558/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:00.672482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539f9d6-a7ee-4fda-82a3-367dcb3e7914 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:00.731425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:00.772256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539f9d6-a7ee-4fda-82a3-367dcb3e7914 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:00.804275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:00.804889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:00.805166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:00.805436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:00.806558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:00.807979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbabdf4-abe2-49c3-bcac-07fc19dc90ad None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:00.809512 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 551/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:35:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:00.815567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:00.911175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747c420-bcaa-4a92-8c39-7539fe02dbf2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:00.943150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:00.943747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:00.944025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:00.944294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:00.944930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:00.946349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570159c7-a8d7-4561-8437-4dfd2041ffb0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:00.968766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 559/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:35:00 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:35:03.414854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1233a-9e84-44d1-b415-3e3083e81d98 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:03.415388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1233a-9e84-44d1-b415-3e3083e81d98 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:03.415388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1233a-9e84-44d1-b415-3e3083e81d98 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:03.416696 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 552/1111] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:03.422436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:03.422846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:03.423033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:03.440730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:03.441913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:03.469696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:03.508488 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 560/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:03.838602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1233a-9e84-44d1-b415-3e3083e81d98 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:03.937128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1233a-9e84-44d1-b415-3e3083e81d98 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:03.961874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:03.970656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:03.971265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:03.971539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:03.971809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] PATH_INFO: `/v3/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:03.973172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] RBAC: Authorizing `identity:get_role(role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:03.974560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:03.975845 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 553/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:03 2022] GET /identity/v3/roles/2413f08d373842418dcfc2305d420a05 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:03.981615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:04.075797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeb16b-ca7d-4b0d-8fe0-0efadbebbf3e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:04.108629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:04.109186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:04.109464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:04.109736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] PATH_INFO: `/v3/users/926afde25647464eb3202f9917391880` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:04.122381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] RBAC: Authorizing `identity:get_user(user_id=926afde25647464eb3202f9917391880)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:04.124514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:04.126092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 561/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:03 2022] GET /identity/v3/users/926afde25647464eb3202f9917391880 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:04.131913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:04.229735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb23a1-4cf2-45d8-bc0b-66aafe8c70a3 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:04.261715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:04.262306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:04.262613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:04.262897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:04.321435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:04.328890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:04.330323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:04.425839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 554/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:04.431789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:04.456353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:04.528261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4f197-d95e-418a-bd8b-a987241b4153 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:04.561176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:04.562108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:04.562364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:04.562674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:04.563794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:04.565225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e072b-3dd3-468a-9063-25c2453ef87f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:04.566878 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 562/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:04 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:04.573156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:04.663978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:04.670860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052004c5-221a-4d93-acef-0c631e0d6adc None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:04.703913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:04.704445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:04.704719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:04.704990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:04.705701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=03fdfc67d1064bc899556d06d686be04, user.id=926afde25647464eb3202f9917391880) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:04.706006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:04.707465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ffd29-10d4-4301-9401-706a1ae73206 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:04.712657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 555/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:04 2022] GET /identity/v3/role_assignments?user.id=926afde25647464eb3202f9917391880&scope.project.id=03fdfc67d1064bc899556d06d686be04&role.id=2413f08d373842418dcfc2305d420a05 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:07.115669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8c0c-7887-41cd-966f-0f685a534a50 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:07.116229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8c0c-7887-41cd-966f-0f685a534a50 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:07.116229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b8c0c-7887-41cd-966f-0f685a534a50 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:07.117526 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 563/1118] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:07.122713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:07.123641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:07.123928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:07.142306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:07.143517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:07.220073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 556/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:07.585805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b8c0c-7887-41cd-966f-0f685a534a50 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:07.691661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:07.704072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b8c0c-7887-41cd-966f-0f685a534a50 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:07.742897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:07.743558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:07.743842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:07.744114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] PATH_INFO: `/v3/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:07.745517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] RBAC: Authorizing `identity:get_role(role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:07.746919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:07.748394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 564/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:07 2022] GET /identity/v3/roles/2413f08d373842418dcfc2305d420a05 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:07.761829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:07.858727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:07.868586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6bc60-ff3e-4115-9882-8c4251db25bf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:07.902718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:07.903349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:07.903629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:07.903909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] PATH_INFO: `/v3/users/926afde25647464eb3202f9917391880` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:07.905712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] RBAC: Authorizing `identity:get_user(user_id=926afde25647464eb3202f9917391880)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:07.907182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:07.908754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 557/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:07 2022] GET /identity/v3/users/926afde25647464eb3202f9917391880 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:07.915985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:08.012490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e43f8-b1f6-41a6-bc42-9ea103ee127f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:08.045672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:08.046380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:08.046690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:08.046965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:08.048059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:08.049475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626065-c70c-46ba-8cda-2b1e9b359eb5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:08.051114 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 565/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:07 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:35:08.057084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:08.153059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfd37f-cda2-47ae-85b8-7977012260f8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:08.188647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:08.189292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:08.189567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:08.189902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04/users/926afde25647464eb3202f9917391880/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:08.192045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] RBAC: Authorizing `identity:create_grant(project_id=03fdfc67d1064bc899556d06d686be04, user_id=926afde25647464eb3202f9917391880, role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:08.193475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa43169-70fc-46d9-a3a0-6372f7def5aa None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:08.206111 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 558/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:35:08 2022] PUT /identity/v3/projects/03fdfc67d1064bc899556d06d686be04/users/926afde25647464eb3202f9917391880/roles/2413f08d373842418dcfc2305d420a05 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:35:10.599124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36519dc-aa52-4fee-9b25-12abdc189716 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:10.599693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36519dc-aa52-4fee-9b25-12abdc189716 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:10.599693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36519dc-aa52-4fee-9b25-12abdc189716 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:10.601027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 566/1124] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:10.606242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:10.606728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:10.606840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:10.625591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:10.626788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:10.639879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:10.694214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 559/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:11.022131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36519dc-aa52-4fee-9b25-12abdc189716 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:11.050611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:11.122486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36519dc-aa52-4fee-9b25-12abdc189716 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:11.155471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:11.156074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:11.156349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:11.156621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] PATH_INFO: `/v3/roles/2413f08d373842418dcfc2305d420a05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:11.157981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] RBAC: Authorizing `identity:get_role(role_id=2413f08d373842418dcfc2305d420a05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:11.159361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:11.160660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 567/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:11 2022] GET /identity/v3/roles/2413f08d373842418dcfc2305d420a05 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:11.166645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:11.263966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650471eb-f1c1-4680-9259-cbca225681ea None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:11.297125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:11.297678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:11.297956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:11.298229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] PATH_INFO: `/v3/users/926afde25647464eb3202f9917391880` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:11.299371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] RBAC: Authorizing `identity:get_user(user_id=926afde25647464eb3202f9917391880)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:11.300807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:11.302353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 560/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:11 2022] GET /identity/v3/users/926afde25647464eb3202f9917391880 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:11.308078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:11.399810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:11.406570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd34ff-92e8-419a-83ac-40f88c10b505 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:11.438778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:11.439368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:11.439641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:11.439908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:11.504709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:11.506133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:11.596068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:11.602674 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 568/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:11.612586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:11.631427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:11.711448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0d987-1a5e-4e52-adb1-516f39a565f9 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:11.743571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:11.744191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:11.744471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:11.744743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] PATH_INFO: `/v3/projects/03fdfc67d1064bc899556d06d686be04` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:11.745840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] RBAC: Authorizing `identity:get_project(project_id=03fdfc67d1064bc899556d06d686be04)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:11.747304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d9198-9da5-4fa3-a4d2-251a13b27503 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:11.749432 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 561/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:11 2022] GET /identity/v3/projects/03fdfc67d1064bc899556d06d686be04 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:11.755746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:11.851298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c5b0f-d66b-4f6e-bfc5-75e923b3fc6d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:11.884831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:11.885422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:11.885701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:11.885973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:11.886715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] RBAC: Adding query filter params (role.id=2413f08d373842418dcfc2305d420a05, scope.project.id=03fdfc67d1064bc899556d06d686be04, user.id=926afde25647464eb3202f9917391880) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:11.887026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:11.888467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91e8cf-2926-4fcc-ba0c-08e9e96fd6b0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:11.894522 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 569/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:11 2022] GET /identity/v3/role_assignments?user.id=926afde25647464eb3202f9917391880&scope.project.id=03fdfc67d1064bc899556d06d686be04&role.id=2413f08d373842418dcfc2305d420a05 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:14.300130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284bf1b-bc34-42b4-8b62-da611d5e9e1e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:14.300660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284bf1b-bc34-42b4-8b62-da611d5e9e1e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:14.300660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284bf1b-bc34-42b4-8b62-da611d5e9e1e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:14.301979 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 562/1131] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:14.307291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:14.307690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:14.307839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:14.325928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:14.327137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:14.379863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:14.391998 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 570/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:14.717041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284bf1b-bc34-42b4-8b62-da611d5e9e1e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:14.799870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:14.816722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284bf1b-bc34-42b4-8b62-da611d5e9e1e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:14.850022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:14.850679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:14.850959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:14.851229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:14.852324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:14.853766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1d6d8-1b49-490f-9de5-33010dd95a6a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:14.855332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 563/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:14.861830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:14.955480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b78c5-1d17-4587-80a9-2352ff812369 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:14.987459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:14.988046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:14.988320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:14.988587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:14.989222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:14.990660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c215ac-fccc-45a1-bdc5-acebf4a88ee4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:15.016569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 571/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:35:14 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:35:17.454290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29bb3-adaf-4dd4-9ee5-24f2cc5e35f3 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:17.454882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29bb3-adaf-4dd4-9ee5-24f2cc5e35f3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:17.454882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29bb3-adaf-4dd4-9ee5-24f2cc5e35f3 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:17.456185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 564/1135] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:17.461572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:17.462018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:17.462120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:17.480767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:17.482757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:17.516201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:17.548506 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 572/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:17.879313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29bb3-adaf-4dd4-9ee5-24f2cc5e35f3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:17.980015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29bb3-adaf-4dd4-9ee5-24f2cc5e35f3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:18.011872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:18.012485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:18.013302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:18.013578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] PATH_INFO: `/v3/domains/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:18.014693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:18.016122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa050b-09a4-4bfd-9348-fb8b87ba0eda None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:18.017641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 565/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:35:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:18.023661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:18.117313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba65121-e813-45fb-8d63-c557d787d7d2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:18.150049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:18.150602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:18.150878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:18.151143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:18.151780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:18.153197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4826c0d-cd34-4ea8-8985-0173d146e246 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:18.174570 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 573/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:35:18 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:35:20.636256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29553df-f2f2-45be-9ad9-db8c9c44373d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:20.636731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29553df-f2f2-45be-9ad9-db8c9c44373d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:20.636840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29553df-f2f2-45be-9ad9-db8c9c44373d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:20.639175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 566/1139] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:20.644657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:20.645064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:20.645196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:20.662989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:20.664161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:20.686596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:20.729824 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 574/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:35:21.054165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29553df-f2f2-45be-9ad9-db8c9c44373d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:21.075818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:21.157322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29553df-f2f2-45be-9ad9-db8c9c44373d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:21.191771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:21.192383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:21.192659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:21.192925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:21.202884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:21.207533 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:21.208699 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:21.209593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 567/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:21.214779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:21.309171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cd68a-6e63-4229-850a-09d439f983c1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:21.341837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:21.342431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:21.342739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:21.343006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:21.343648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:21.343942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:21.345276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:21.352838 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 575/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:21.359450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:21.438082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:21.456637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358ed9a-5fb5-4d79-a7e0-ceccb365f870 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:21.489593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:21.490209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:21.490485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:21.490782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] PATH_INFO: `/v3/users/fb20c407886a454db74b782726043122` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:21.504091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] RBAC: Authorizing `identity:get_user(user_id=fb20c407886a454db74b782726043122)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:21.505532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:21.507164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 568/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/users/fb20c407886a454db74b782726043122 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:21.513151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:21.535450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:21.611882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e37a79-74df-46a7-895d-5fc3c64f32e7 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:21.646207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:21.646850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:21.647131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:21.647404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:21.711327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:21.712764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:21.732572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:21.807376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 576/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:21.813588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:21.910221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450aa62-dfa6-4cb1-844e-dde276042e15 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:21.942974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:21.943586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:21.943863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:21.944136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] PATH_INFO: `/v3/projects/1856dd848a6a4612ad43c258e60cd940` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:21.945239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] RBAC: Authorizing `identity:get_project(project_id=1856dd848a6a4612ad43c258e60cd940)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:21.946693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619fc2f-7080-4a43-ac1c-bbc5d0aa12ca None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:21.948291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 569/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/projects/1856dd848a6a4612ad43c258e60cd940 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:21.955097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:22.049667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a1567-f4a0-498e-8c21-c74bdf7e0a1d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:22.069084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:22.083459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:22.084046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:22.084326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:22.084627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:22.085334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=1856dd848a6a4612ad43c258e60cd940, user.id=fb20c407886a454db74b782726043122) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:22.085639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:22.087091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d24a5-d196-4ed0-b8de-e35962f800c0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:22.092566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 577/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:21 2022] GET /identity/v3/role_assignments?user.id=fb20c407886a454db74b782726043122&scope.project.id=1856dd848a6a4612ad43c258e60cd940&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:24.454819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503ab0-f759-41d8-a8d9-574debe5f498 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:24.455351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503ab0-f759-41d8-a8d9-574debe5f498 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:24.455351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e503ab0-f759-41d8-a8d9-574debe5f498 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:24.456675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 570/1147] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:24.461903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:24.462311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:24.462444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:24.480742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:24.482570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:24.546129 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 578/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:24.876446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e503ab0-f759-41d8-a8d9-574debe5f498 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:24.885281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:24.975019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e503ab0-f759-41d8-a8d9-574debe5f498 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:25.007557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:25.008168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:25.008447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:25.008718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:25.018842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:25.023524 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:25.024685 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:25.025530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 571/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:25.031250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:25.125140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c6243-ed63-4ebd-be5f-1652973eade2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:25.138306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:25.158744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:25.159341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:25.159622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:25.159894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:25.160544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:25.160845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:25.162198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:25.170612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 579/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:25.176322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:25.244602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:25.278709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45bcb-6429-44dc-9b74-bac906f2d71b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:25.311830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:25.312444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:25.312720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:25.312986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] PATH_INFO: `/v3/users/fb20c407886a454db74b782726043122` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:25.314078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] RBAC: Authorizing `identity:get_user(user_id=fb20c407886a454db74b782726043122)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:25.315543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:25.317085 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 572/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:25 2022] GET /identity/v3/users/fb20c407886a454db74b782726043122 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:25.322844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:25.420456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441236-8f3e-4c0a-85dd-ca5959e6a082 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:25.452490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:25.453092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:25.453367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:25.453635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] PATH_INFO: `/v3/projects/1856dd848a6a4612ad43c258e60cd940` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:25.454740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] RBAC: Authorizing `identity:get_project(project_id=1856dd848a6a4612ad43c258e60cd940)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:25.456153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada80202-c202-45ed-bbde-bc1504e91252 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:25.457733 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 580/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:25 2022] GET /identity/v3/projects/1856dd848a6a4612ad43c258e60cd940 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:25.463763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:25.559323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a19a3-a676-4c8e-b143-f9fb71ee71e5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:25.592903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:25.593546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:25.593820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:25.594101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] PATH_INFO: `/v3/projects/1856dd848a6a4612ad43c258e60cd940/users/fb20c407886a454db74b782726043122/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:25.596025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] RBAC: Authorizing `identity:create_grant(project_id=1856dd848a6a4612ad43c258e60cd940, user_id=fb20c407886a454db74b782726043122, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:25.597447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e88199-ee6e-4f4c-b073-f81120efc699 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:25.599700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:25.610233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 573/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:35:25 2022] PUT /identity/v3/projects/1856dd848a6a4612ad43c258e60cd940/users/fb20c407886a454db74b782726043122/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:35:27.993366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78066f2-3561-4a16-86f2-742a453f8c79 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:27.993868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78066f2-3561-4a16-86f2-742a453f8c79 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:27.993868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78066f2-3561-4a16-86f2-742a453f8c79 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:27.995207 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 581/1154] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:28.001206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:28.001626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:28.001745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:28.020472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:28.021635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:28.085877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 574/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:28.420495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78066f2-3561-4a16-86f2-742a453f8c79 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:28.466247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:28.520577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78066f2-3561-4a16-86f2-742a453f8c79 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:28.552864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:28.553474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:28.553751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:28.554022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:28.564389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:28.569109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:28.570121 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:35:28.570973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 582/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:28.576245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:28.677168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe57dc-5431-43d6-aee3-e1bf06854930 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:28.700711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:28.711979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:28.712569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:28.712841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:28.713104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:28.713781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:28.714073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:28.716253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:28.723909 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 575/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:28.729954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:28.826672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f12f3d-e64c-48e0-ba3e-80853f0c1111 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:28.828912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:28.860218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:28.860827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:28.861098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:28.861362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] PATH_INFO: `/v3/users/fb20c407886a454db74b782726043122` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:28.862447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] RBAC: Authorizing `identity:get_user(user_id=fb20c407886a454db74b782726043122)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:28.863896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:28.865440 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 583/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:28 2022] GET /identity/v3/users/fb20c407886a454db74b782726043122 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:28.871386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:28.972950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36427a95-228d-4f57-863d-afa5a2add93d None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:29.004888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:29.005481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:29.005730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:29.005996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:29.057824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:29.070756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:29.072192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:29.167447 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 576/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:29.173451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:29.275146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9a4e-8d03-4ace-9e69-6d6b8fd655dd None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:29.306980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:29.307593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:29.307870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:29.308168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] PATH_INFO: `/v3/projects/1856dd848a6a4612ad43c258e60cd940` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:29.309336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] RBAC: Authorizing `identity:get_project(project_id=1856dd848a6a4612ad43c258e60cd940)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:29.310831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16680e48-0012-4413-80ae-acfa52365190 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:29.312436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 584/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:29 2022] GET /identity/v3/projects/1856dd848a6a4612ad43c258e60cd940 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:29.318660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:29.399889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:29.415348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400119f7-566d-4b37-aa4d-50a8de6f73e5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:29.448387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:29.448975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:29.449299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:29.449571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:29.450277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=1856dd848a6a4612ad43c258e60cd940, user.id=fb20c407886a454db74b782726043122) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:29.450607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:29.452027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359cf565-0094-4c78-94e4-e72ed4e63505 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:29.457485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 577/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:29 2022] GET /identity/v3/role_assignments?user.id=fb20c407886a454db74b782726043122&scope.project.id=1856dd848a6a4612ad43c258e60cd940&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:31.888335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c10fb-37c2-4c46-ba88-cff679813008 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:31.888859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c10fb-37c2-4c46-ba88-cff679813008 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:31.888859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c10fb-37c2-4c46-ba88-cff679813008 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:31.890102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 585/1162] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:31.895552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:31.895957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:31.896097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:31.914988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:31.916184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:31.981017 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 578/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:35:32.311474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c10fb-37c2-4c46-ba88-cff679813008 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:32.353271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:32.409438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c10fb-37c2-4c46-ba88-cff679813008 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:32.442146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:32.442764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:32.443042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:32.443309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:32.506368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:32.507814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:32.545702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:32.601712 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 586/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:32.607805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:32.690567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:32.709559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776202be-c23b-437d-a7b2-3eae88d7d72e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:32.742073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:32.742717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:32.742996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:32.743268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:32.750133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:32.751601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-87090373-c559-452c-ae6b-de03a33bae87 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:32.763556 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:32.764790 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:32.765680 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 579/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:35:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:32.771014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:32.865775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd5ecf-8e7d-479f-81ab-180724a79adc None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:32.898840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:32.899436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:32.899694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:32.899966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:32.900591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:32.900891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:32.902272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa625192-9e82-464f-ad90-071e72595c50 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:32.909137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 587/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Sat Sep 24 20:35:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:35.303549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93253082-1655-4d48-af72-4f40d29e5fa0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:35.304104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93253082-1655-4d48-af72-4f40d29e5fa0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:35.304104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93253082-1655-4d48-af72-4f40d29e5fa0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:35.305400 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 580/1167] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:35.310976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e91bc-e86b-4095-841a-f50349df89ad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:35.311423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e91bc-e86b-4095-841a-f50349df89ad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:35.311722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e91bc-e86b-4095-841a-f50349df89ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:35.319326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:35.331687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-962e91bc-e86b-4095-841a-f50349df89ad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:35.332853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-962e91bc-e86b-4095-841a-f50349df89ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:35.397544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 588/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:35.726239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93253082-1655-4d48-af72-4f40d29e5fa0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:35.825311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93253082-1655-4d48-af72-4f40d29e5fa0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:35.857178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:35.857768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:35.858043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:35.858311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:35.859002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:35.860375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835b239-4ae6-44a9-b271-1d48631dee19 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:35.870318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:35.878095 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 581/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Sat Sep 24 20:35:35 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:35:38.315624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfaf9b-40cd-42f2-bda7-6087609cbc27 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:38.316177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfaf9b-40cd-42f2-bda7-6087609cbc27 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:38.316177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfaf9b-40cd-42f2-bda7-6087609cbc27 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:38.317491 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 589/1170] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:38.322822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:38.323215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:38.323370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:38.342391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:38.343601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:38.408345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 582/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:38.733189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfaf9b-40cd-42f2-bda7-6087609cbc27 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:38.832269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfaf9b-40cd-42f2-bda7-6087609cbc27 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:38.864388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:38.864998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:38.865270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:38.865538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] PATH_INFO: `/v3/domains/6eabf3461f5548a5a47afbf1f3b93f21` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:38.867357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:38.876798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6eabf3461f5548a5a47afbf1f3b93f21)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:38.878232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3ee12-7dfa-44ce-adc3-7124af507252 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:38.879824 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 590/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Sat Sep 24 20:35:38 2022] GET /identity/v3/domains/6eabf3461f5548a5a47afbf1f3b93f21 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:38.885659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:38.980828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0c6f9-353c-4df8-b0da-9a73cbbbbbcc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:39.008790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:39.014952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:39.015547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:39.015826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:39.016096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:39.016731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:39.018417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9c5ee-b173-4363-a8d2-c3e261f8be18 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:39.044840 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 583/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:35:38 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:35:41.469379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962af3-08e7-4429-80e9-4f7ae70c2a66 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:41.469933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962af3-08e7-4429-80e9-4f7ae70c2a66 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:41.469933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962af3-08e7-4429-80e9-4f7ae70c2a66 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:41.471289 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 591/1174] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:41.476589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:41.477029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:41.477128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:41.497236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:41.498400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:41.564314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 584/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:41.893200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62962af3-08e7-4429-80e9-4f7ae70c2a66 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:41.990401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62962af3-08e7-4429-80e9-4f7ae70c2a66 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:42.023056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:42.023665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:42.023944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:42.024216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] PATH_INFO: `/v3/domains/6eabf3461f5548a5a47afbf1f3b93f21` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:42.025313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6eabf3461f5548a5a47afbf1f3b93f21)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:42.027348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d817953-339c-4828-ad34-7df3d6e95ff5 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:42.028884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 592/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Sat Sep 24 20:35:41 2022] GET /identity/v3/domains/6eabf3461f5548a5a47afbf1f3b93f21 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:42.034546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:42.129343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4122c73-a911-4cc5-b9ca-91128490b794 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:42.142859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:42.164444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:42.164972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:42.165244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:42.165510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:42.166136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:42.167881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6256151-d36c-4df5-86e9-79f702f32a6a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:42.187886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 585/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:35:42 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:35:44.629171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa46c-c564-4e1a-883a-a77c5c18e2f9 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:44.629745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa46c-c564-4e1a-883a-a77c5c18e2f9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:44.629745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfa46c-c564-4e1a-883a-a77c5c18e2f9 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:44.631103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 593/1178] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:44.636741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:44.637162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:44.637285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:44.656064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:44.657256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:44.726400 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 586/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:45.054104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfa46c-c564-4e1a-883a-a77c5c18e2f9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:45.113986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:45.151889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfa46c-c564-4e1a-883a-a77c5c18e2f9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:45.183387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:45.183988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:45.184260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:45.184524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:45.194084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:45.199322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:45.200513 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:35:45.201604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 594/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:45.206510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:45.298750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:45.302143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665abc24-de76-41b3-8445-dc46cffbc42a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:45.337141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:45.337732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:45.338010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:45.338281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:45.339186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:45.339491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:45.340865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:45.348930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 587/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:45.355136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:45.458487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b617ed-c6d1-4709-9fae-180b19c6db40 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:45.479879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:45.491349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:45.491967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:45.492244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:45.492513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] PATH_INFO: `/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:45.506126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] RBAC: Authorizing `identity:get_user(user_id=8eaab4cbc4df4a57a53b6a75fea2e4ce)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:45.507582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:45.509153 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 595/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:45.515973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:45.613465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51fa9-e657-4c8e-be4d-c7e34698c1bf None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:45.647863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:45.648464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:45.648738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:45.649006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:45.676179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:45.716343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:45.717793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:45.812444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 588/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:45.818436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:45.922086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c6b97-3900-42f0-be3d-e59a0661d04a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:45.954597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:45.955206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:45.955479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:45.955743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] PATH_INFO: `/v3/projects/9f65e373060e4f67a5afa90c4757711d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:45.956824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] RBAC: Authorizing `identity:get_project(project_id=9f65e373060e4f67a5afa90c4757711d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:45.958224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3424c0-8ebc-4c6b-981e-9f580a9686a0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:45.959832 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 596/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/projects/9f65e373060e4f67a5afa90c4757711d => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:45.967157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:46.023668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:46.062790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c057193-a6c3-479c-bdfa-dc79cfb62e4f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:46.095316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:46.095891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:46.096168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:46.096433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:46.097128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=9f65e373060e4f67a5afa90c4757711d, user.id=8eaab4cbc4df4a57a53b6a75fea2e4ce) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:46.097425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:46.098860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8583c-dfa9-4072-83cb-c846ec607b70 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:46.104148 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 589/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:45 2022] GET /identity/v3/role_assignments?user.id=8eaab4cbc4df4a57a53b6a75fea2e4ce&scope.project.id=9f65e373060e4f67a5afa90c4757711d&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:48.474187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbd45-d61f-4535-898e-902cbf68173d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:48.474856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbd45-d61f-4535-898e-902cbf68173d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:48.474856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdbd45-d61f-4535-898e-902cbf68173d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:48.476101 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 597/1186] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:48.481897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:48.482305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:48.482415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:48.500035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:48.501212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:48.565299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 590/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:48.891646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdbd45-d61f-4535-898e-902cbf68173d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:48.940997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:48.990948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdbd45-d61f-4535-898e-902cbf68173d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:49.023621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:49.024235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:49.024513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:49.024783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:49.035063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:49.039783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:49.040866 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:35:49.042622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 598/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:35:49.047634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:49.100154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:49.144808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68ea53-1072-4be3-b487-4d98105d62b5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:49.178353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:49.178968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:49.179246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:49.179515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:49.180168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:49.180467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:49.181829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:49.189797 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 591/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:49.195692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:49.292540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acfa97-1274-4517-a845-2462028672f5 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:49.301267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:49.325739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:49.326347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:49.326657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:49.326933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] PATH_INFO: `/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:49.328029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] RBAC: Authorizing `identity:get_user(user_id=8eaab4cbc4df4a57a53b6a75fea2e4ce)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:49.329443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:49.331185 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 599/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:49 2022] GET /identity/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:49.336693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:49.434771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91420ce3-cae0-42e4-aa0a-840176040e01 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:49.450429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:49.468511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:49.469126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:49.469401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:49.469671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] PATH_INFO: `/v3/projects/9f65e373060e4f67a5afa90c4757711d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:49.470791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] RBAC: Authorizing `identity:get_project(project_id=9f65e373060e4f67a5afa90c4757711d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:49.472215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba82079-02e0-4838-b409-ad3b2438062c None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:49.473800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 592/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:49 2022] GET /identity/v3/projects/9f65e373060e4f67a5afa90c4757711d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:49.479845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:49.574350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167658ac-d2b8-45fb-9ddc-cf8c628b56c8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:49.606076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:49.606735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:49.607011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:49.607280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] PATH_INFO: `/v3/projects/9f65e373060e4f67a5afa90c4757711d/users/8eaab4cbc4df4a57a53b6a75fea2e4ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:49.609112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] RBAC: Authorizing `identity:create_grant(project_id=9f65e373060e4f67a5afa90c4757711d, user_id=8eaab4cbc4df4a57a53b6a75fea2e4ce, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:49.610533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da046627-16b4-4fa9-8cfc-5356c61023eb None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:49.623297 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 600/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:35:49 2022] PUT /identity/v3/projects/9f65e373060e4f67a5afa90c4757711d/users/8eaab4cbc4df4a57a53b6a75fea2e4ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:35:52.022276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50960bf-c6fd-49c1-9f01-103b0ab09948 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:52.022804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50960bf-c6fd-49c1-9f01-103b0ab09948 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:52.022804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50960bf-c6fd-49c1-9f01-103b0ab09948 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:52.024066 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 593/1193] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:35:52.030097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:52.030556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:52.030700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:52.048894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:35:52.050050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:35:52.077054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:52.115164 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 601/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:35:52.443532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50960bf-c6fd-49c1-9f01-103b0ab09948 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:52.542382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50960bf-c6fd-49c1-9f01-103b0ab09948 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:52.574567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:52.575178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:52.575452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:52.575721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:52.585840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:35:52.592225 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:35:52.593339 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:35:52.594190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 594/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:35:52.599346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:52.694484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6518d6-035f-474b-af95-952fc6e0784f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:52.728163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:52.728963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:52.729308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:52.729592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:52.730302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:52.730628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:52.732169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:52.740113 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 602/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:52.746953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:52.792369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:52.847669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1b5c-0810-4c15-b0fe-5d5eebe41f03 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:52.879980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:52.880586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:52.880859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:52.881126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] PATH_INFO: `/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:52.882213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] RBAC: Authorizing `identity:get_user(user_id=8eaab4cbc4df4a57a53b6a75fea2e4ce)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:52.883670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:52.886145 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 595/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Sat Sep 24 20:35:52 2022] GET /identity/v3/users/8eaab4cbc4df4a57a53b6a75fea2e4ce => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:52.892141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:52.919439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:52.987479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef435ae-655e-4fa0-81cc-ce535594c727 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:53.020095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:53.020682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:53.020955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:53.021221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:53.085054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:53.086472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:53.112808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:53.184712 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 603/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:35:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:35:53.190636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:53.286533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe12805-3e81-4afb-9ffb-f570551bb767 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:53.299016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:53.319982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:53.320599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:53.320876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:53.321145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] PATH_INFO: `/v3/projects/9f65e373060e4f67a5afa90c4757711d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:53.322268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9f65e373060e4f67a5afa90c4757711d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:53.323981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93ee0e-74f0-40b1-915b-951284d422a7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:53.325727 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 596/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Sat Sep 24 20:35:53 2022] GET /identity/v3/projects/9f65e373060e4f67a5afa90c4757711d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:35:53.331654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:35:53.433353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7720afe-ac5c-4b96-b29f-88a53dd55755 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:35:53.458217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:35:53.467401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:35:53.467993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:35:53.468271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:35:53.468541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:35:53.469243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=9f65e373060e4f67a5afa90c4757711d, user.id=8eaab4cbc4df4a57a53b6a75fea2e4ce) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:35:53.469548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:35:53.470995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba4be-b3ff-4873-a1cb-22809178cec1 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:35:53.476496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 604/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:35:53 2022] GET /identity/v3/role_assignments?user.id=8eaab4cbc4df4a57a53b6a75fea2e4ce&scope.project.id=9f65e373060e4f67a5afa90c4757711d&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:36:03.316561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:36:03.473966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:36:57.234786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5a05e-c791-42ce-b7ea-bcbf22e9f1be None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:36:57.235453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5a05e-c791-42ce-b7ea-bcbf22e9f1be None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:36:57.235453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5a05e-c791-42ce-b7ea-bcbf22e9f1be None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:36:57.236668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 597/1201] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:36:57.242296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:36:57.243183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:36:57.243753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:36:57.265217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:36:57.266636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:36:57.355227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 605/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:36:57.751236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5a05e-c791-42ce-b7ea-bcbf22e9f1be None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:36:57.890354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5a05e-c791-42ce-b7ea-bcbf22e9f1be None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:36:57.937576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:36:57.937576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:36:57.937576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:36:57.937576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:36:57.949974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:36:57.951483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:36:57.966898 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:36:57.968037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:36:57.969008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:36:57.969008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:36:57.969008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:36:57.969008 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 598/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:36:57.980791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:36:58.129090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be724ce6-0065-453e-842d-e47074da1ed1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:36:58.170557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:36:58.177122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:36:58.177801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:36:58.178131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:36:58.178406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:36:58.182067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:36:58.182067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:36:58.182067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4197fe-c8d5-4f50-9271-490b2b7c7b9f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:36:58.195928 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 606/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:36:58.202411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:36:58.298325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:36:58.370186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495040b-a793-4456-a9f7-7ec207f4ec7b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:36:58.424350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:36:58.424873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:36:58.425150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:36:58.425419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:36:58.426079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:36:58.427556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd36fc-89c4-44cd-9ea0-3e61bf207e78 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:36:58.458523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 599/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Sat Sep 24 20:36:58 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:37:01.267928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53e3b-34e7-4847-8d7f-3d7ea066f383 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:01.270090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53e3b-34e7-4847-8d7f-3d7ea066f383 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:01.270090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa53e3b-34e7-4847-8d7f-3d7ea066f383 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:01.270586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 607/1206] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:01.277559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:01.278075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:01.278183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:01.300308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:01.301691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:01.384907 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 600/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:01.717125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa53e3b-34e7-4847-8d7f-3d7ea066f383 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:01.832968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa53e3b-34e7-4847-8d7f-3d7ea066f383 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:01.868386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:01.869165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:01.869591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:01.869867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:01.879950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:01.885582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:01.886614 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:01.887473 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:01.891643 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 608/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:37:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:01.892929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:01.935471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:02.010413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a29d73-3629-4d5a-8c55-8ddaeb8e177f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:02.052254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:02.052893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:02.053170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:02.053438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:02.054140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:02.054437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:02.055948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:02.065010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 601/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:37:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:02.071403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:02.131150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:02.203866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937c2f-2126-4100-848c-3a13619601ee None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:02.248768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:02.250656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:02.251285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:02.252162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:02.267252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:02.269699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:02.290032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:02.291889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:02.293017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:02.293017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:02.293017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:02.300717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 609/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:37:02.301669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:02.422237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:02.425573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e7363-1913-481b-8bfe-6b77563559ae None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:02.466299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:02.467049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:02.467422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:02.467804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:02.468539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:02.468945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:02.470565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:02.484469 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 602/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:02.490828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:02.610948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91690d-7172-43e2-8666-04da7fb8ab9d None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:02.612840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:02.654465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:02.655361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:02.655710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:02.656146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:02.665641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:02.667347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:02.681149 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:02.682922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:02.684193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 610/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:02.688780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:02.821421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11fa16-207f-4dfb-914a-3ae0ec2d4da5 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:02.872314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:02.872916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:02.873196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:02.873468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:02.874199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:02.874516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:02.875975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:02.898648 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 603/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:02.900676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:03.017761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168b5c-23de-4bf8-8e6b-bfef322f6c81 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:03.061823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:03.062703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:03.063089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:03.063375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:03.100690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:03.152306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:03.153894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:03.299148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 611/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:37:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:37:03.305126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:03.326859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:03.419111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d93d9-b404-4665-a6cb-5cbadb761654 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:03.455321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:03.455897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:03.456239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:03.456477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:03.468492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:03.470210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:03.486917 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:03.487912 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:03.488847 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:03.488847 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:03.488847 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:03.488847 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 604/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:03.495773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:03.564674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:03.608325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778115e-92fb-4ed9-9f94-fdf3e04b4859 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:03.645418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:03.646227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:03.646486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:03.646799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:03.647470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:03.647775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:03.649298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:03.662267 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 612/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:03.668654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:03.752150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:03.794565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ceba4-b3c3-4e53-b74b-3bea0c4db849 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:03.841118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:03.841773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:03.842120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:03.842394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:03.850629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:03.852070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:03.865879 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:03.867034 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:03.868118 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 605/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:03.875760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:04.001977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2e1-894e-477c-a7de-d8b813d31fe4 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:04.025405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:04.040236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:04.041534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:04.042121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:04.042121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:04.042843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:04.043096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:04.044861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbaaa8-63e6-454e-9143-19848e689388 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:04.064879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:04.065305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 613/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:04.190141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f465092-0d62-4541-a49f-7ac43e9f6be6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:04.228941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:04.236412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:04.237089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:04.237366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:04.237636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:04.238390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=441c7f4882d74eaabbe434654e9d862d) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:04.238790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:04.240378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a868c-bedc-4b79-8693-6de9e7f06662 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:04.250040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 606/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:37:04 2022] GET /identity/v3/role_assignments?user.id=441c7f4882d74eaabbe434654e9d862d&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:06.779698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb419-742d-4e62-abc0-b8f03338d187 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:06.780420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb419-742d-4e62-abc0-b8f03338d187 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:06.780420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb419-742d-4e62-abc0-b8f03338d187 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:06.781586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 614/1220] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:06.787244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:06.787699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:06.787811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:06.812480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:06.813916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:06.890361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 607/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:07.230575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb419-742d-4e62-abc0-b8f03338d187 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:07.341323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb419-742d-4e62-abc0-b8f03338d187 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:07.375847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:07.376443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:07.376720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:07.377038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:07.387549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:07.392360 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:07.394823 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:07.395990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 615/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:07.399741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:07.516925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e81c2-e9a8-41ea-beb1-5966b8e4a1e6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:07.544658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:07.556242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:07.558221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:07.558522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:07.559030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:07.559744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] RBAC: Adding query filter params (name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:07.560098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:07.561610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:07.570489 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 608/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:07.576893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:07.623873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:07.683826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135af8a9-33f0-4ec2-8b6e-5f98cf8393a6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:07.720124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:07.720771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:07.721044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:07.721313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:07.733669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:07.737149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:07.754143 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:07.755561 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:07.756466 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:07.756466 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:07.756466 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:07.756466 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 616/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:37:07.762138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:07.874036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e042-97c3-4f41-9cda-d0495d5b5dea None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:07.914298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:07.915008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:07.915331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:07.915670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:07.917121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:07.917517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:07.918964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:07.931779 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 609/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:07.938660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.041835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:08.049221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23955f10-f42d-448c-b963-ee1f6d775d7a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.084800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.085487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.085768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.086036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] PATH_INFO: `/v3/users/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.094933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.096492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:08.110085 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:08.111291 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:08.112671 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:08.113504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 617/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:08.118898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.168379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:08.225039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafa-239f-459e-a6de-c70a211fc0a1 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.272782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.273512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.273851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.274303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.275204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:08.276068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.277651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:08.299225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 610/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:08.305190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.421180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dfcdd-4cd2-49ac-843c-9ac7aea34f9c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.461033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.461715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.462026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.462294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.474440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.476002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:08.490480 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:08.491506 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:08.492460 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:08.492460 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:08.492460 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:08.492460 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 618/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:08.498468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.610289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db59ce8-6e8e-4ed3-988b-3763892a9c4b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.612605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:08.649394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.649994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.650266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.650652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.651289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:08.651582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.652943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:08.664849 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 611/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:08.671099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.680495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:08.776124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ce201-b579-463e-9793-ce0322cda3ef None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.811345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.812106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.812418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.812720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.820169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.821699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:08.833680 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:08.840268 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:08.841531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 619/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:08.842080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:08.949769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdd606-c629-4aed-82e3-686b2d05f156 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:08.984769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:08.985366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:08.985640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:08.985906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:08.986610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:08.986914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:08.988335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e43fe1-d318-49db-95e6-41f75a7a5d6a None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:09.000938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 612/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:09.006912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:09.076215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:09.113999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c759c9-bda0-43cf-ace9-698ba8eaf263 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:09.148539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:09.149194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:09.149515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:09.149812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/441c7f4882d74eaabbe434654e9d862d/roles/7f9068477c864176b24e4ffaac5ad69f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:09.165310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=441c7f4882d74eaabbe434654e9d862d, role_id=7f9068477c864176b24e4ffaac5ad69f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:09.167341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e513c-68d4-4b74-9300-d6d778e3b955 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:09.179900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 620/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:37:09 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/441c7f4882d74eaabbe434654e9d862d/roles/7f9068477c864176b24e4ffaac5ad69f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:37:11.721514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34315-8385-47db-8d9c-fb0bc044a0db None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:11.722185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34315-8385-47db-8d9c-fb0bc044a0db None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:11.722185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f34315-8385-47db-8d9c-fb0bc044a0db None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:11.723401 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 613/1233] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:11.729867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:11.730430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:11.730755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:11.752941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:11.754379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:11.834809 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 621/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:12.211598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34315-8385-47db-8d9c-fb0bc044a0db None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:12.244273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:12.345777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f34315-8385-47db-8d9c-fb0bc044a0db None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:12.391383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:12.392015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:12.392289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:12.392762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:12.405660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:12.411067 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:12.414670 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:12.415598 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 614/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Sat Sep 24 20:37:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:12.418325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:12.474183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:12.542650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36fcc-8a2d-41df-a566-3a24c7b32b78 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:12.580127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:12.580732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:12.581016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:12.581287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:12.581981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:12.582261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:12.583657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:12.592191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 622/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Sat Sep 24 20:37:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:12.598147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:12.692571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:12.709995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6daaba-b818-4df7-8b85-db0d05c143ed None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:12.747377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:12.748217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:12.748217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:12.748453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:12.758699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:12.760189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:12.775290 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:12.776954 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:12.777993 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:12.777993 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:12.777993 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:12.777993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 615/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:12.783911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:12.883185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200951-c2e0-47f1-959e-b6292bfda3f3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:12.892207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:12.922537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:12.922537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:12.922537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:12.922537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:12.922975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:12.922975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:12.922975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:12.935902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 623/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:12.941666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:13.052366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3a286-8dd5-401c-a359-84ec4b05c55e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:13.087519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:13.088721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:13.089076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:13.089440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:13.092367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:13.097711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:13.099536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:13.112368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:13.113660 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:13.115007 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 616/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:13.119491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:13.233210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096a45-9e6d-4185-aef9-c478d10eb627 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:13.282833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:13.283600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:13.283959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:13.284279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:13.285151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:13.285486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:13.287421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:13.307858 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 624/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:13.314459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:13.427241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abbdd7-ca80-420f-9320-399be6299d67 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:13.464829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:13.465414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:13.465700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:13.466010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:13.536048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:13.544366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:13.546242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:13.663264 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 617/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:37:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:37:13.669265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:13.698550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:13.775352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68f8b3-9737-4e48-8f76-1328a8a1b005 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:13.811384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:13.812064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:13.812347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:13.812645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:13.823952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:13.825398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:13.841450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:13.842869 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:13.844200 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:13.844200 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:13.844200 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:13.847704 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 625/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:13.848833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:13.957821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:13.965262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3bcbc-9d92-4e93-ba4b-c2c6b5a1982b None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:14.003935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:14.004534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:14.004813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:14.005100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:14.005768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:14.006022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:14.007409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:14.025549 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 618/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:14.026860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:14.112124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:14.140482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af879ab6-bfb9-4698-90eb-fc81d7c8080a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:14.175742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:14.176370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:14.176643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:14.176910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:14.183851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:14.185352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:14.196826 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:14.197948 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:14.199150 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 626/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:14.203994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:14.328385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee141d6-5e97-406a-bb60-f6c19629c0c6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:14.368551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:14.369305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:14.369819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:14.370190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:14.370952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:14.371302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:14.372891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8452ff4-db32-4929-8295-5f23f35e1499 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:14.387168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 619/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:14.393747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:14.506217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054b19b-9056-4d7c-adb9-10f6e992304a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:14.545236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:14.546105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:14.546559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:14.546858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:14.547701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] RBAC: Adding query filter params (role.id=7f9068477c864176b24e4ffaac5ad69f, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=441c7f4882d74eaabbe434654e9d862d) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:14.548055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:14.549626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d762f0-6135-4a03-9aa3-bee74348b4ba None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:14.557587 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 627/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:37:14 2022] GET /identity/v3/role_assignments?user.id=441c7f4882d74eaabbe434654e9d862d&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=7f9068477c864176b24e4ffaac5ad69f => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:17.112020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b13d5-8c95-4fc1-8c76-0fec29ceedeb None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:17.112556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b13d5-8c95-4fc1-8c76-0fec29ceedeb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:17.112556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279b13d5-8c95-4fc1-8c76-0fec29ceedeb None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:17.113852 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 620/1247] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:17.119182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:17.120203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:17.120343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:17.126585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:17.139793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:17.140990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:17.208601 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 628/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:17.538880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b13d5-8c95-4fc1-8c76-0fec29ceedeb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:17.556584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:17.638652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279b13d5-8c95-4fc1-8c76-0fec29ceedeb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:17.671572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:17.672187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:17.672462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:17.672728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:17.682576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:17.687251 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:17.688321 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:17.689179 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 621/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:17.694332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:17.789574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093f65f-2e44-4c87-9796-f095cffa721a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:17.818046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:17.824187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:17.824789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:17.825070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:17.825373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:17.826025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:17.826327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:17.827736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:17.835822 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 629/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:17.842153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:17.917559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:17.940992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1e64a-9d2b-418a-92e6-6e3808676c1c None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:17.974442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:17.975078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:17.975352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:17.975619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:17.985494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:17.986965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:18.001696 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:18.002852 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:18.003810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:18.003810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:18.003810 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:18.003810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 622/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:18.008516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:18.103741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8b2a3-c9dc-4af9-bde9-39c6e862fee6 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:18.136566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:18.137142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:18.137419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:18.137690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:18.138312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:18.138645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:18.139995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.152042 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 630/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:18.157697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:18.263934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64611e73-3bf9-4a76-b7b0-5d9c76c28f47 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:18.289935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:18.297875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:18.298491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:18.298797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:18.299068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:18.306183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:18.307870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:18.318583 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:18.319682 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:18.321005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 623/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:18.326278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:18.378801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:18.422653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d81226-7c07-4fb3-9760-d5f8ed7386e0 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:18.454664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:18.455250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:18.455526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:18.455794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:18.456494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:18.456795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:18.458204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.474999 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 631/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:18.480899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:18.577427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9571239-dac9-4f82-98a8-abdfb6d0e584 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:18.610849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:18.611438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:18.611712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:18.611981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:18.667687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:18.680038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:18.681496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.776137 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 624/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:37:18.781889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:18.877755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29e9ed-0c68-43dd-97d8-49808c2635db None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:18.890857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:18.911585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:18.912203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:18.912481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:18.912754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:18.922595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:18.924040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:18.938582 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:18.944159 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:18.945378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:18.945378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:18.945378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:18.945378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 632/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:18.945657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:19.030708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:19.043159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac474-51c8-4d4d-9852-c6547fd0ed8e None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:19.076512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:19.077106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:19.077385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:19.077660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:19.078291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:19.078624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:19.080038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:19.091185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 625/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:19.096717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:19.195303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b15ee-391e-423a-b736-d862e1a7759a None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:19.229901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:19.230810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:19.231089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:19.231383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:19.238678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:19.240350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:19.254128 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:19.255335 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:19.256500 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 633/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:19.262040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:19.359717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb0a3-f04e-4a07-9d95-e0c283ef10a0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:19.391786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:19.392379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:19.392654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:19.392924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:19.393630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:19.393932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:19.395396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472054-ef7b-4a57-b6ad-c12ad8d507bd None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:19.407601 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 626/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:19.413627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:19.439314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:19.511321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d3011-a43c-4a92-81b3-51506bd47bc1 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:19.543819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:19.544406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:19.544679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:19.544945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:19.545639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=441c7f4882d74eaabbe434654e9d862d) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:19.545939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:19.547378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bcca4-8037-4f38-8548-c4d518eca033 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:19.552449 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 634/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:37:19 2022] GET /identity/v3/role_assignments?user.id=441c7f4882d74eaabbe434654e9d862d&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:21.908395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ee93-7bdb-4e9d-a52b-a0e0698642a6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:21.908897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ee93-7bdb-4e9d-a52b-a0e0698642a6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:21.908897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ee93-7bdb-4e9d-a52b-a0e0698642a6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:21.910146 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 627/1261] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:21.915271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:21.915648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:21.915825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:21.934141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:21.935457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:21.998726 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 635/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:22.335168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10ee93-7bdb-4e9d-a52b-a0e0698642a6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:22.346614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:22.435643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10ee93-7bdb-4e9d-a52b-a0e0698642a6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:22.469119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:22.469749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:22.470032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:22.470304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:22.480381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:22.485120 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:22.486193 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:22.487164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 628/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:22.492195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:22.510230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:22.588647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47792eea-76ef-44e8-952c-7a153bf3383a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:22.622286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:22.622917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:22.623190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:22.623454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:22.624095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:22.624389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:22.625738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:22.634073 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 636/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:22.640184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:22.705507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:22.737222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9484-4702-4ca9-8da7-50d005cab6ea None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:22.771021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:22.771623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:22.771896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:22.772162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:22.781753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:22.783215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:22.797667 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:22.798800 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:22.799873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:22.799873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:22.799873 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:22.799873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 629/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:22.804462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:22.883248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:22.901908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fce5b9-ac8b-4a30-a656-9a4c7d9e3b41 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:22.934356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:22.935004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:22.935254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:22.935524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:22.936149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:22.936449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:22.937794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:22.949043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 637/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:22.954875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.050596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ffbb-0fce-4b05-89d0-d15fed7118b7 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.069565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:23.083344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.083953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.084228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.084497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.091761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.093189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:23.103914 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:23.105725 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:23.107004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 630/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:23.111370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.206262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28a23-0124-4104-b72d-671e06e37116 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.242486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.243106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.243400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.243654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.244365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:23.244668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.246355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.254957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:23.263664 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 638/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:23.269384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.366439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf901da3-0edb-45a4-90cb-cba612e8a0b8 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.398563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.399180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.399453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.399718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.410428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.411887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:23.426093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:23.427190 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:23.428487 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:23.428487 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:23.428487 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:23.428487 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 631/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:23.432924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.529059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933603e4-566b-4d04-8b1d-761cc275d9a8 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.561274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.561865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.562141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.562407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.563063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:23.563358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.564698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.576087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 639/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:23.581774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.612540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:23.681310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58adcff8-46d5-41b4-8c2d-a6bba6044428 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.714684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.715295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.715569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.715838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.722191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.723731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:23.734983 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:23.736003 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:23.737212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 632/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 20:37:23.741847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.798294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:23.840500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15665353-df1b-4e1a-be36-5ddc930863ad None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:23.872751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:23.873340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:23.873614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:23.873879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:23.874575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:23.874879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:23.876279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba5347-fd72-4057-a858-40b58cf6d4a8 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:23.888520 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 640/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:23.894478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:23.984636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:23.993402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e41c-45fd-4c94-9cb2-eae775414a62 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:24.026863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:24.027496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:24.027773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:24.028043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] PATH_INFO: `/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/441c7f4882d74eaabbe434654e9d862d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:24.029915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e368838cf0844c799bbf8bbd5b5a803, user_id=441c7f4882d74eaabbe434654e9d862d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:24.031385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b908843-95f1-40a1-be58-59f1a8aaddbb None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:24.043880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 633/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Sat Sep 24 20:37:23 2022] PUT /identity/v3/projects/8e368838cf0844c799bbf8bbd5b5a803/users/441c7f4882d74eaabbe434654e9d862d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:37:26.408664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e8eb-51cd-4bd6-b74a-61d9ae2ba597 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:26.409198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e8eb-51cd-4bd6-b74a-61d9ae2ba597 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:26.409198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e8eb-51cd-4bd6-b74a-61d9ae2ba597 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:26.410617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 641/1274] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:26.415766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:26.416144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:26.416312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:26.434804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:26.435988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:26.500804 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 634/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:26.825171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e8eb-51cd-4bd6-b74a-61d9ae2ba597 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:26.922579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e8eb-51cd-4bd6-b74a-61d9ae2ba597 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:26.939316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:26.956064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:26.956673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:26.956950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:26.957218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:26.966999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 20:37:26.971684 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:26.972777 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:26.974028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 642/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Sat Sep 24 20:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:26.978702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.056947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:27.076472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339195f9-0087-4edb-85dd-701be6b487e8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.108907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.109502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.109780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.110052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.110734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:27.111030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.112396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:27.120535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 635/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Sat Sep 24 20:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:27.126452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.223757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc60ed7-20b3-4985-b855-6959f0f87b50 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.259719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.261130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.261461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.261795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.275393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.277753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:27.293536 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:27.294809 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:27.295889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:27.295889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:27.295889 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:27.295889 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 643/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:27.300918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.398082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275984d5-4791-4f5e-90c8-ad5aa13a05e6 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.430995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.431606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.431849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.432117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.432776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:27.433073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.434447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:27.445736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 636/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:27.452175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.482430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:27.549591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702c68-f731-4081-8af9-a978f0739e50 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.581936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.582576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.582852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.583118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] PATH_INFO: `/v3/users/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.590651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.592055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 20:37:27.602702 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:27.603802 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:27.605129 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 644/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:27.609985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.613811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:27.711320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f323f9d-31b4-4a4d-8988-c95b3dc41e67 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.743933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.744556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.744832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.745698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.746310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:27.746642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.748073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:27.764945 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 637/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Sat Sep 24 20:37:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:27.770861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:27.857360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:27.866926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eecc5-fac2-463f-acc5-cde3dd08195c None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:27.899672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:27.900256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:27.900532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:27.900799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:27.964249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:27.965613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.049946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:28.060405 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 645/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Sat Sep 24 20:37:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 20:37:28.066200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:28.125792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:28.164947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a18f2-a9a4-4f94-a27e-b2dd930cb688 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:28.198928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:28.199548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:28.199826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:28.200098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:28.209706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:28.211180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 20:37:28.226284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:28.227368 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 20:37:28.228688 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 20:37:28.228688 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 20:37:28.228688 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:28.228688 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 638/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:28.233265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:28.327776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a0f8a-3970-4af8-a70b-0f76c049047f None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:28.359771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:28.360355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:28.360626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:28.360890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:28.361505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:28.361800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:28.363157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.373727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 646/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Sat Sep 24 20:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:28.379379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:28.476073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e65f2-6907-4419-b608-1ca847810796 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:28.489169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:28.509034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:28.509639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:28.509912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:28.510181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:28.517039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:28.518462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 20:37:28.529448 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:28.530570 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 20:37:28.531838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 639/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Sat Sep 24 20:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:28.536333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:28.585726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:28.634093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d808cf-3ecd-4838-99cc-c977e5d84177 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:28.668298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:28.668882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:28.669154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:28.669416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:28.670086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:28.670384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:28.671832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335201-9775-47a6-a608-df41e9af9c53 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.683744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 647/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Sat Sep 24 20:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:28.689789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:28.787130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69bff9-21ab-48dc-a047-237055730f5f None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:28.819969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:28.820553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:28.820827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:28.821095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:28.821805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] RBAC: Adding query filter params (role.id=49b89b74bd2a42b6ba0c6496570ab8dc, scope.project.id=8e368838cf0844c799bbf8bbd5b5a803, user.id=441c7f4882d74eaabbe434654e9d862d) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:28.822110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:28.823570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330dc2f2-a43b-4920-a8bc-aa7ba248f400 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:28.829162 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 640/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Sat Sep 24 20:37:28 2022] GET /identity/v3/role_assignments?user.id=441c7f4882d74eaabbe434654e9d862d&scope.project.id=8e368838cf0844c799bbf8bbd5b5a803&role.id=49b89b74bd2a42b6ba0c6496570ab8dc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:31.210123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66841-6390-4069-bdcc-1e3d3d17db3c None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:31.210687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66841-6390-4069-bdcc-1e3d3d17db3c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:31.210687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca66841-6390-4069-bdcc-1e3d3d17db3c None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:31.212007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 648/1288] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:31.217807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:31.218252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:31.218345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:31.237903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:31.239298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:31.273025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:31.312684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 641/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:31.645786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66841-6390-4069-bdcc-1e3d3d17db3c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:31.744818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca66841-6390-4069-bdcc-1e3d3d17db3c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:31.755801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:31.779127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:31.779743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:31.780020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:31.780289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:31.780857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:31.782204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:31.790620 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:31.791778 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:31.793934 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 649/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:37:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:31.798436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:31.894985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315afbc-a03e-4cc6-826e-92d1955d0448 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:31.928122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:31.928715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:31.928995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:31.929266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:31.929887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:31.930188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:31.931586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc5d30-3c70-40a4-a322-c5d5cae4af28 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:31.936367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 642/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 20:37:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:31.941969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:32.037836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b01cd-d84c-4a72-9a33-84445eeb8ce3 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:32.069214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:32.069793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:32.070068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:32.070343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:32.070976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] RBAC: Adding query filter params (type=placement) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:32.071276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:32.072616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cad47-215e-4dfe-bf5f-f549d59e718d None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:32.077335 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 650/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 20:37:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:34.501658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ddb-8c78-4e8d-9f1f-fe2c18994e7c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:34.502191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ddb-8c78-4e8d-9f1f-fe2c18994e7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:34.502191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d7ddb-8c78-4e8d-9f1f-fe2c18994e7c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:34.503548 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 643/1293] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:34.508547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f50df-0a9a-4323-9d5e-04b7dc7b7ded None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:34.508952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f50df-0a9a-4323-9d5e-04b7dc7b7ded None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:34.509095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f50df-0a9a-4323-9d5e-04b7dc7b7ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:34.527686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e59f50df-0a9a-4323-9d5e-04b7dc7b7ded None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:34.528862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e59f50df-0a9a-4323-9d5e-04b7dc7b7ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:34.556637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:34.594110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 651/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:34.924948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d7ddb-8c78-4e8d-9f1f-fe2c18994e7c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:34.955811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:35.024494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d7ddb-8c78-4e8d-9f1f-fe2c18994e7c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:35.058175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:35.058883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:35.059154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:35.059416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:35.060053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:35.061405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e12e8-60ce-4f2d-bb2f-43636bdd9df7 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:35.075541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 644/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Sat Sep 24 20:37:34 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:37:37.462976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bfbdb-9b78-463a-bf5b-592f431f360a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:37.463547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bfbdb-9b78-463a-bf5b-592f431f360a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:37.463547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bfbdb-9b78-463a-bf5b-592f431f360a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:37.465504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 652/1296] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:37.470744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:37.471148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:37.471289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:37.489658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:37.490872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:37.561246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 645/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 20:37:37.890010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bfbdb-9b78-463a-bf5b-592f431f360a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:37.988010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bfbdb-9b78-463a-bf5b-592f431f360a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:38.020717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:38.021334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:38.021614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:38.022150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:38.022752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:38.024103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:38.033246 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:38.034413 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:38.035560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 653/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:37:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:38.040423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:38.058130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:38.136548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5078498-f098-4435-8f55-cbf49c25e109 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:38.170017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:38.170572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:38.170854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:38.171126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:38.171747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:38.172045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:38.173409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:38.178281 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 646/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 20:37:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:38.184868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:38.216033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:38.283406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efa778-6eb2-4527-a934-6330a1e84186 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:38.315723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:38.316315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:38.316595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:38.316865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:38.317548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] RBAC: Adding query filter params (interface=public, service_id=110f291b6944460f9f5d8abca8b1a358, region_id=RegionOne) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:38.317853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:38.319242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5587894-af87-4344-a7da-8595a8d8bbb7 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:38.324757 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 654/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Sat Sep 24 20:37:38 2022] GET /identity/v3/endpoints?service_id=110f291b6944460f9f5d8abca8b1a358&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:37:38.330409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:38.408775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:38.426775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35f45-76f6-44a0-b4b9-11c110f2bdd0 None admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:38.459919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:38.460510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:38.460787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:38.461055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:38.461642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:38.463036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ff498-6c1e-4620-aa56-4b5067c26035 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:38.469533 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 647/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Sat Sep 24 20:37:38 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:37:40.864414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d07d-bdc6-48ac-a173-8301ec4c62e7 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:40.864987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d07d-bdc6-48ac-a173-8301ec4c62e7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:40.864987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5d07d-bdc6-48ac-a173-8301ec4c62e7 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:40.866283 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 655/1302] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:37:40.871668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:40.872105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:40.872208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:40.890538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:37:40.891655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:37:40.956332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 648/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:37:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 20:37:41.284808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5d07d-bdc6-48ac-a173-8301ec4c62e7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:41.379799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:41.386944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5d07d-bdc6-48ac-a173-8301ec4c62e7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:41.420420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:41.421038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:41.421314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:41.421582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] PATH_INFO: `/v3/services/placement` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:41.422143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:41.423527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 20:37:41.431958 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 20:37:41.433426 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 20:37:41.434580 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 20:37:41.438081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 656/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Sat Sep 24 20:37:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 20:37:41.439082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:41.514403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:41.537993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2974-2067-4ed3-8ddd-250ea9e6a3dd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:41.570474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:41.571101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:41.571381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:41.571653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:41.572271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] RBAC: Adding query filter params (name=placement) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:37:41.572572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:41.574524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a37714-027d-4d46-bddf-9c33df376a55 None admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:41.579488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 649/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Sat Sep 24 20:37:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:37:41.585174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:37:41.684400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfe919-8307-4d10-9add-0fc53cc85e93 None admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:37:41.716916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:37:41.717505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:37:41.717778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:37:41.718045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:37:41.718723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:37:41.720066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3d83d-aa6b-409d-8b6c-72d26c2c933f None admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:37:41.738827 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 657/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:37:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:37:51.576541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=30 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:37:51.734296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:38:55.020152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4296f-df3a-4dd6-afcc-73dcf21042f2 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:55.021009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4296f-df3a-4dd6-afcc-73dcf21042f2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:55.021009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4296f-df3a-4dd6-afcc-73dcf21042f2 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:55.022217 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 650/1307] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:38:55.027590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0f79-3dc4-414f-a2e7-a6b2a7d3abb8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:55.028051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0f79-3dc4-414f-a2e7-a6b2a7d3abb8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:55.028337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df0f79-3dc4-414f-a2e7-a6b2a7d3abb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:55.213238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-41df0f79-3dc4-414f-a2e7-a6b2a7d3abb8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:38:55.214522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-41df0f79-3dc4-414f-a2e7-a6b2a7d3abb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:55.362881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 658/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:38:55.715118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:55.715466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:55.715676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:55.718124 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 651/1309] 10.222.0.52 () {54 vars in 964 bytes} [Sat Sep 24 20:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:38:55.725809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8f5b-4c02-4899-96ce-1dbee49aec2a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:55.726117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8f5b-4c02-4899-96ce-1dbee49aec2a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:55.726564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8f5b-4c02-4899-96ce-1dbee49aec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:55.947569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f30e8f5b-4c02-4899-96ce-1dbee49aec2a None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:38:55.948810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f30e8f5b-4c02-4899-96ce-1dbee49aec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:55.972242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:38:56.032271 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 659/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 20:38:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:38:56.044730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:38:56.047190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:56.174403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351348-0ca0-44b1-adc3-c14301444928 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:38:56.261099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:38:56.262484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:56.263732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:56.264265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:56.266328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:56.332805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:38:56.411524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:38:56.413653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8e534-bfdd-4a80-be47-7f3d4a679740 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:38:56.455428 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 652/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:38:59.251974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:59.252512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:59.252512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:59.254186 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 660/1312] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:38:59.259827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0743-bca4-4e07-bfeb-afbb2d9fe8d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:59.260265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0743-bca4-4e07-bfeb-afbb2d9fe8d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:59.260361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f0743-bca4-4e07-bfeb-afbb2d9fe8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:59.334593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f59f0743-bca4-4e07-bfeb-afbb2d9fe8d1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:38:59.335803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f59f0743-bca4-4e07-bfeb-afbb2d9fe8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:59.434338 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 653/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:38:59.784301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:38:59.785908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:38:59.888141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b04007-ac69-4db2-b7be-828c02e3ab39 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:38:59.963514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:38:59.965567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:38:59.966374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:38:59.967169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:38:59.967687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:38:59.969901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:00.087842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:00.089305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d72bf-7071-48e7-8f8c-acfb420def69 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:00.110202 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 661/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:02.842403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:02.843009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:02.843009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:02.844299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 654/1315] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:02.849480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81005e-5c52-454b-8232-1a1e66c39a81 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:02.849882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81005e-5c52-454b-8232-1a1e66c39a81 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:02.850069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81005e-5c52-454b-8232-1a1e66c39a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:02.918099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5c81005e-5c52-454b-8232-1a1e66c39a81 None None] MFA Rules not processed for user `1fb4d47a50b04a298d4a8d7db55b1da4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:02.920484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c81005e-5c52-454b-8232-1a1e66c39a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:02.971966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:03.015565 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 662/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:03.358663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:03.360184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:03.370067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:03.464273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e208e-42d4-444d-89a4-ccd87daa2c15 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:03.536670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:03.537268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:03.537545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:03.537819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:03.539441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:03.600198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:03.647597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:03.649054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0da42-0f1c-45df-9e11-6c02d47af7d0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:03.667501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 655/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:06.605458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:06.606025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:06.606025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:06.607944 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 663/1318] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:06.613370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3efb01-4f9b-4556-9b14-efa55acb3a27 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:06.613784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3efb01-4f9b-4556-9b14-efa55acb3a27 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:06.613940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3efb01-4f9b-4556-9b14-efa55acb3a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:06.680095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9f3efb01-4f9b-4556-9b14-efa55acb3a27 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:06.681277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9f3efb01-4f9b-4556-9b14-efa55acb3a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:06.774349 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 656/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:07.117800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:07.119368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:07.228618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6118d6-a2e4-481c-8465-866f954f42fe None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:07.304329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:07.304885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:07.305140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:07.305408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:07.307058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:07.314024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:07.414632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:07.416049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717198a-03c7-4f37-9160-dc383470c6e1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:07.433973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 664/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:10.519429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:10.519969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:10.520086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:10.521357 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 657/1321] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:10.526759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f662-773e-4dbe-a5b2-199ce6bd80b0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:10.527417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f662-773e-4dbe-a5b2-199ce6bd80b0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:10.527570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f662-773e-4dbe-a5b2-199ce6bd80b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:10.594594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cd39f662-773e-4dbe-a5b2-199ce6bd80b0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:10.595795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd39f662-773e-4dbe-a5b2-199ce6bd80b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:10.657142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:10.696348 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 665/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:11.037668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:11.039415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:11.059994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:11.145665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad61afa-9f32-4cf9-a16a-4dc788574e07 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:11.222274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:11.222907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:11.223186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:11.223458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:11.225051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:11.296789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:11.335608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:11.337052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52977a0a-76c9-42da-90f9-db377ed5ab1d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:11.354972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 658/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:14.116099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:14.116689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:14.116689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:14.117984 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 666/1324] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:14.124029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3074-4615-460b-bb19-ed11dd786653 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:14.124497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3074-4615-460b-bb19-ed11dd786653 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:14.124676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de3074-4615-460b-bb19-ed11dd786653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:14.191276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c4de3074-4615-460b-bb19-ed11dd786653 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:14.192482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c4de3074-4615-460b-bb19-ed11dd786653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:14.297402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 659/1325] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:14.646380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:14.647922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:14.751963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842e249-b2d7-43ef-a3dc-4027cdd90a8e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:14.825679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:14.826302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:14.826671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:14.826949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:14.828520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:14.845984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:14.938995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:14.940358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4bf2e-86e4-452c-a60f-5f1a3e59bf3e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:14.959753 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 667/1326] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:17.464750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:17.465314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:17.465314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:17.467248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 660/1327] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:17.472785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6726-a8fa-4993-aa80-08aed4288ea7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:17.473217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6726-a8fa-4993-aa80-08aed4288ea7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:17.473334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba6726-a8fa-4993-aa80-08aed4288ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:17.539385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0ba6726-a8fa-4993-aa80-08aed4288ea7 None None] MFA Rules not processed for user `1fb4d47a50b04a298d4a8d7db55b1da4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:17.540575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0ba6726-a8fa-4993-aa80-08aed4288ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:17.604657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:17.638754 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 668/1328] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:17.985241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:17.986863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:18.020861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:18.097419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddbed4-4b00-4379-a549-fd0be88cd3d1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:18.173161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:18.173753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:18.174026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:18.174294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:18.175892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:18.256718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:18.289913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:18.291397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d30cf-32fc-4a7e-9c5f-7c38295532ee service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:18.314013 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 661/1329] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:20.988622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9916-bdf4-475b-9af8-b1e495d27398 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:20.989293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9916-bdf4-475b-9af8-b1e495d27398 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:20.989293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9916-bdf4-475b-9af8-b1e495d27398 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:20.991673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 669/1330] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:20.996236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf514569-8725-4889-933e-b40d8f141d93 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:20.996661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf514569-8725-4889-933e-b40d8f141d93 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:20.996767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf514569-8725-4889-933e-b40d8f141d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:21.076928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bf514569-8725-4889-933e-b40d8f141d93 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:21.078128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf514569-8725-4889-933e-b40d8f141d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:21.186572 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 662/1331] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:21.560236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:21.560515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:21.560660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:21.562381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 670/1332] 10.222.0.52 () {54 vars in 964 bytes} [Sat Sep 24 20:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:21.568819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e9a1-7a9f-4a54-8415-afcf3d9dbc73 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:21.569105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e9a1-7a9f-4a54-8415-afcf3d9dbc73 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:21.569402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506e9a1-7a9f-4a54-8415-afcf3d9dbc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:21.617273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:21.655837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e506e9a1-7a9f-4a54-8415-afcf3d9dbc73 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:21.657036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e506e9a1-7a9f-4a54-8415-afcf3d9dbc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:21.735835 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 663/1333] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 20:39:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:21.744881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:21.746401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:21.851845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089652f0-8a2d-4518-86cb-9ea1634b9fa0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:21.930020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:21.930652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:21.930924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:21.931184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:21.932787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:21.961117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:22.053234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:22.054726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d6d97-313d-4fbf-b48a-05cfd865548e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:22.074042 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 671/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:24.770716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:24.771242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:24.771242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:24.772575 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 664/1335] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:24.777964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676862-d0ee-4496-a7f2-1c207a6ecfc7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:24.778933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676862-d0ee-4496-a7f2-1c207a6ecfc7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:24.779066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2676862-d0ee-4496-a7f2-1c207a6ecfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:24.844277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f2676862-d0ee-4496-a7f2-1c207a6ecfc7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:24.845491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2676862-d0ee-4496-a7f2-1c207a6ecfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:24.918745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:24.941274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 672/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:25.286953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:25.288505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:25.393502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb21d6-3423-4a35-bb77-6391c8053464 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:25.423242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:25.468437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:25.469059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:25.469340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:25.469618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:25.471318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:25.583220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:25.584674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b6cfd6-d446-403a-b9e6-d0059d858734 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:25.602290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 665/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:28.609949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:28.610487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:28.610487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:28.611853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 673/1338] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:28.617054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5eb-a496-47cc-b5d5-f54ea58b5057 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:28.617480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5eb-a496-47cc-b5d5-f54ea58b5057 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:28.617599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5eb-a496-47cc-b5d5-f54ea58b5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:28.684000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-70f6c5eb-a496-47cc-b5d5-f54ea58b5057 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:28.685204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-70f6c5eb-a496-47cc-b5d5-f54ea58b5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:28.693297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:28.781616 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 666/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:29.119216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:29.120749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:29.221974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47c6f0-8df0-4d28-b1ab-32c2934ff314 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:29.294924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:29.295521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:29.295795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:29.296058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:29.297654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:29.333122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:29.405504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:29.406898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4325e8-b758-45e7-abb4-226db95ce4bf service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:29.424449 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 674/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:33.924086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:33.924653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:33.924653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:33.925933 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 667/1341] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:33.931385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919cb0-9d78-46d2-9955-e63da78a2dca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:33.931816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919cb0-9d78-46d2-9955-e63da78a2dca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:33.932060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd919cb0-9d78-46d2-9955-e63da78a2dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:33.999332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fd919cb0-9d78-46d2-9955-e63da78a2dca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:34.000328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd919cb0-9d78-46d2-9955-e63da78a2dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:34.093212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:34.104684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 675/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:34.458018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:34.459584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:34.566937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480544f-2222-4bf1-abe7-b234ecec3fdd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:34.605469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:34.645709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:34.646312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:34.646620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:34.646891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:34.648478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:34.758765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:34.760206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220c3d4-f678-4d02-8e4d-f2e5cce19754 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:34.779551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 668/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:37.820309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:37.820900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:37.820900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:37.822167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 676/1344] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:37.827709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360abb-d1b7-4e62-8500-acfd07ecf73f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:37.828109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360abb-d1b7-4e62-8500-acfd07ecf73f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:37.828263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360abb-d1b7-4e62-8500-acfd07ecf73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:37.894804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fb360abb-d1b7-4e62-8500-acfd07ecf73f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:37.895908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fb360abb-d1b7-4e62-8500-acfd07ecf73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:37.914589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:37.993380 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 669/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:39:38.336795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:38.338317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:38.444003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b8353-b435-4748-a8f4-56e251ba4fe6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:38.517945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:38.518580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:38.518857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:38.519122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:38.520710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:38.565332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:38.635764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:38.637105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b353f63-721e-4f91-b896-34fbc4eab477 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:38.657315 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 677/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:43.039573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:43.039982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:43.040132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:43.041478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 670/1347] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:43.047444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6818-fd90-4b44-8412-2461fab4235a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:43.047874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6818-fd90-4b44-8412-2461fab4235a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:43.048023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6818-fd90-4b44-8412-2461fab4235a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:43.115162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-807a6818-fd90-4b44-8412-2461fab4235a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:43.116538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-807a6818-fd90-4b44-8412-2461fab4235a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:43.215020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:43.218533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 678/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:43.556048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:39:43.557589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:43.666298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14da00a-429c-47c8-9745-4d521b7a61f8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:39:43.715656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:43.742923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:39:43.743443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:43.743713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:43.743979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:43.745570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:43.860907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:39:43.862360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b0846-3ae0-4180-afcb-3654692ad3fe service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:39:43.880865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 671/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:39:49.390463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d415c-672b-4ad7-8ea2-46265d8dd022 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:49.391063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d415c-672b-4ad7-8ea2-46265d8dd022 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:49.391063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d415c-672b-4ad7-8ea2-46265d8dd022 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:49.392636 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 679/1350] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:49.397744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebd033-6205-49cf-834d-af48e85fdec0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:49.399008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebd033-6205-49cf-834d-af48e85fdec0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:49.399363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebd033-6205-49cf-834d-af48e85fdec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:49.483582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1cebd033-6205-49cf-834d-af48e85fdec0 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:49.484899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1cebd033-6205-49cf-834d-af48e85fdec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:49.506267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:49.663142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 672/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:39:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:54.050393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001e86-2b95-41c2-bd31-09247d5a5858 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:54.050946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001e86-2b95-41c2-bd31-09247d5a5858 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:54.051096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001e86-2b95-41c2-bd31-09247d5a5858 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:54.052662 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 680/1352] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:54.060676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbad00-cb56-466a-900c-044283396392 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:54.061134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbad00-cb56-466a-900c-044283396392 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:54.061347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbad00-cb56-466a-900c-044283396392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:54.148555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-88bbad00-cb56-466a-900c-044283396392 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:54.149775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-88bbad00-cb56-466a-900c-044283396392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:54.234371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:39:54.278636 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 673/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:39:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:55.853069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933a5f-cfb1-47b4-be6d-d7056ae7a21f None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:55.853581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933a5f-cfb1-47b4-be6d-d7056ae7a21f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:55.853711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62933a5f-cfb1-47b4-be6d-d7056ae7a21f None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:55.855351 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 681/1354] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:55.860743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324fba7-f6ad-4f00-8a7b-be257536947f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:55.861166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324fba7-f6ad-4f00-8a7b-be257536947f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:55.861433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324fba7-f6ad-4f00-8a7b-be257536947f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:55.939517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a324fba7-f6ad-4f00-8a7b-be257536947f None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:39:55.940761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a324fba7-f6ad-4f00-8a7b-be257536947f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:39:56.056745 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 674/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:39:59.949404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660312-ccc2-45e9-a0e1-6763ec7112f3 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:59.949977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660312-ccc2-45e9-a0e1-6763ec7112f3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:59.951015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad660312-ccc2-45e9-a0e1-6763ec7112f3 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:59.952599 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 682/1356] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:39:59.957933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102f6cd-39b8-4a3e-abe7-4209222efd60 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:39:59.958397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102f6cd-39b8-4a3e-abe7-4209222efd60 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:39:59.958481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102f6cd-39b8-4a3e-abe7-4209222efd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:39:59.994659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:00.027587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c102f6cd-39b8-4a3e-abe7-4209222efd60 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:00.029263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c102f6cd-39b8-4a3e-abe7-4209222efd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:00.133772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 675/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:00.494801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:00.495504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:00.496265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:00.499763 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 683/1358] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:40:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:00.505439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5cc5-0d9f-428d-82ad-5263aa6b0a3a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:00.505734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5cc5-0d9f-428d-82ad-5263aa6b0a3a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:00.506019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d5cc5-0d9f-428d-82ad-5263aa6b0a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:00.604258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2c7d5cc5-0d9f-428d-82ad-5263aa6b0a3a None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:00.605459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2c7d5cc5-0d9f-428d-82ad-5263aa6b0a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:00.670804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:00.749937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 676/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:00.758231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:00.759753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:00.915448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bad82-32ef-437b-bfe6-f7653552ffca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:00.991610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:01.034693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:01.035301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:01.035583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:01.035858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:01.037455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:01.133364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:01.134991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4f3df-e958-4181-bf62-3a55165215b2 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:01.155774 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 684/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:40:03.818263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:03.818841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:03.818841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:03.820155 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 677/1361] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:03.825836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd989e-55f8-46ca-ad7b-3a13ef2358b4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:03.826420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd989e-55f8-46ca-ad7b-3a13ef2358b4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:03.826420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd989e-55f8-46ca-ad7b-3a13ef2358b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:03.911751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-19cd989e-55f8-46ca-ad7b-3a13ef2358b4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:03.913313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19cd989e-55f8-46ca-ad7b-3a13ef2358b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:03.958090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:04.023595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 685/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:04.376796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:04.378363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:04.545168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ce2d1-aa0e-47ca-a181-9a77e36a556f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:04.554433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:04.672786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:04.673407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:04.673683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:04.673952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:04.675638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:04.777406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:04.779350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d56ae-f786-41ec-b73b-398168b331a9 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:04.802822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 678/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:07.582636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1232a1-366c-4cd6-a4b3-de75108de743 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:07.583855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1232a1-366c-4cd6-a4b3-de75108de743 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:07.585242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1232a1-366c-4cd6-a4b3-de75108de743 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:07.592152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 686/1364] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:07.593394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad75df-3930-429b-81be-246f4353aea2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:07.593831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad75df-3930-429b-81be-246f4353aea2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:07.594013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad75df-3930-429b-81be-246f4353aea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:07.613815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:07.723943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-30ad75df-3930-429b-81be-246f4353aea2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:07.725085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30ad75df-3930-429b-81be-246f4353aea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:07.836616 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 679/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:08.192309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:08.192840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:08.193267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:08.195081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 687/1366] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:08.202666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05873-519e-4824-bfe6-c0feec347d32 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:08.202953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05873-519e-4824-bfe6-c0feec347d32 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:08.203230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05873-519e-4824-bfe6-c0feec347d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:08.316641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12a05873-519e-4824-bfe6-c0feec347d32 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:08.318128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12a05873-519e-4824-bfe6-c0feec347d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:08.361385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:08.476241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:08.477885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:08.478672 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 680/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:08.696204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ae8f9-cf14-4d59-9f24-1a8db3c142b5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:08.733068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:08.875297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:08.887223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:08.887223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:08.887223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:08.887223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:08.993909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:08.995690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8286f2f-0b39-4406-8013-7c89ba0e929a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:09.017326 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 688/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:40:11.865608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:11.867862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:11.868386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:11.875535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488dbf-23c6-428b-a508-4f280e9e1ebe None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:11.876473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488dbf-23c6-428b-a508-4f280e9e1ebe None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:11.876973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488dbf-23c6-428b-a508-4f280e9e1ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:11.882108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 681/1369] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:11 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:11.915306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:11.959857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-98488dbf-23c6-428b-a508-4f280e9e1ebe None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:11.961570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98488dbf-23c6-428b-a508-4f280e9e1ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:12.099887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 689/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:12.480972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:12.482836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:12.733589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:12.747701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad98a57-f8c2-41f1-9686-20c0e79c8b9f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:12.884249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:12.885894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:12.885894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:12.885894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:12.887135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:12.987945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:12.989472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555265ec-be3b-4b02-bd7c-d77e850ea928 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:13.006806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:13.009436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 682/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:13.115030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f1841-c853-46c8-a4be-04bf92b575e5 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:13.115674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f1841-c853-46c8-a4be-04bf92b575e5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:13.115674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f1841-c853-46c8-a4be-04bf92b575e5 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:13.117140 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 690/1372] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:13.123786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9edc-20e9-4d9a-8141-5df27d30e00c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:13.124807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9edc-20e9-4d9a-8141-5df27d30e00c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:13.125306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9edc-20e9-4d9a-8141-5df27d30e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:13.224056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ab5a9edc-20e9-4d9a-8141-5df27d30e00c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:13.225939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ab5a9edc-20e9-4d9a-8141-5df27d30e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:13.350777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 683/1373] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:13.930056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:13.930056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:13.930056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:13.931006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 691/1374] 10.222.0.52 () {54 vars in 963 bytes} [Sat Sep 24 20:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:13.936558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c3f5-aa5e-4a1b-9305-d2d84d897bf4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:13.937078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c3f5-aa5e-4a1b-9305-d2d84d897bf4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:13.937078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2c3f5-aa5e-4a1b-9305-d2d84d897bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:14.017097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2ce2c3f5-aa5e-4a1b-9305-d2d84d897bf4 None None] MFA Rules not processed for user `c66ad10d7496429da3b81648cf9e83f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:14.018288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2ce2c3f5-aa5e-4a1b-9305-d2d84d897bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:14.040787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:14.105445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 684/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Sat Sep 24 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:14.113788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:14.115439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:14.231796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9afc5b-0c22-4d9a-a06a-df21738df59b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:14.236048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:14.318057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:14.318797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:14.319301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:14.319570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:14.321144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:14.462452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:14.464303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516cb47-e4b4-4730-b05e-3e67177ba938 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:14.496717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 692/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:15.853761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:15.854417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:15.854417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:15.855772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 685/1377] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:15.861437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1449d-25a7-4805-92d9-cd9485f2be53 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:15.861927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1449d-25a7-4805-92d9-cd9485f2be53 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:15.862084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1449d-25a7-4805-92d9-cd9485f2be53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:15.909466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:15.937350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d4f1449d-25a7-4805-92d9-cd9485f2be53 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:15.938655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4f1449d-25a7-4805-92d9-cd9485f2be53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:16.046532 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 693/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:16.407508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:16.409086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:16.565647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53e638-7e94-4a2d-85b2-40615c4eec64 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:16.594321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:16.685440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:16.686122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:16.686405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:16.686721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:16.688349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:16.786079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:16.787734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ee2f0-2641-409f-81de-11371f65d90e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:16.805968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 686/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:19.279843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:19.281078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:19.281460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:19.286547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 694/1380] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:40:19.288253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbebebe-200a-4f0b-8194-dd6ae60dfc49 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:19.288694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbebebe-200a-4f0b-8194-dd6ae60dfc49 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:19.288785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbebebe-200a-4f0b-8194-dd6ae60dfc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:19.331042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:19.358240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6fbebebe-200a-4f0b-8194-dd6ae60dfc49 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:19.359467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6fbebebe-200a-4f0b-8194-dd6ae60dfc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:19.487369 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 687/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:19.835638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:19.837512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:19.952421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:19.993942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c311d-465f-4814-bdbc-b8401375f7c3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:20.121058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:20.121877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:20.122216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:20.122733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:20.124593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:20.203421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:20.220025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:20.221664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0aa81-3c3a-4aee-8f9c-cf719d94d1a9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:20.241145 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 695/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:22.758932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:22.759429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:22.759429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:22.760802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 688/1383] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:22.766841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c3b0-208b-48cc-981c-e76e119987e4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:22.767304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c3b0-208b-48cc-981c-e76e119987e4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:22.767490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29c3b0-208b-48cc-981c-e76e119987e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:22.851629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3b29c3b0-208b-48cc-981c-e76e119987e4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:22.853236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b29c3b0-208b-48cc-981c-e76e119987e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:22.970243 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 696/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:23.457514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:23.459115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:23.588929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:23.647133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f894e57-6c8a-492b-b822-2ac0c4fcb658 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:23.781714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:23.782926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:23.782926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:23.782926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:23.784479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:23.876908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:23.895459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:23.897095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b560fa-a0ea-4691-a644-7bb18364b288 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:23.917116 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 689/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:40:26.595994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:26.596741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:26.596741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:26.597969 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 697/1386] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:26.603636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248119-7b2b-4a76-98f6-9e9a4397836e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:26.604105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248119-7b2b-4a76-98f6-9e9a4397836e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:26.604209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248119-7b2b-4a76-98f6-9e9a4397836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:26.691782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d2248119-7b2b-4a76-98f6-9e9a4397836e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:26.693089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d2248119-7b2b-4a76-98f6-9e9a4397836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:26.812027 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 690/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:27.305899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:27.307596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:27.371746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:27.477573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe2a53-1730-4375-a9d8-f5bca8c2853a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:27.609620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:27.610357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:27.610574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:27.610858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:27.612528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:27.627153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:27.712757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:27.714358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e995ec-8a13-459c-a1ae-fdaeb403a384 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:27.739255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 698/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:30.418107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:30.422459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:30.422459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:30.422459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 691/1389] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:30.425602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505b7dc-4fd8-4c97-8c3c-34507bafdfca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:30.426051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505b7dc-4fd8-4c97-8c3c-34507bafdfca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:30.426150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505b7dc-4fd8-4c97-8c3c-34507bafdfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:30.492377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9505b7dc-4fd8-4c97-8c3c-34507bafdfca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:30.493612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9505b7dc-4fd8-4c97-8c3c-34507bafdfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:30.584136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:30.596859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 699/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:30.949727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:30.951970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:31.004437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:31.051258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0352-b494-4e34-8bcc-1297da24ea4b None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.051543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0352-b494-4e34-8bcc-1297da24ea4b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.051841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c0352-b494-4e34-8bcc-1297da24ea4b None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.053186 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 700/1391] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:40:31.058041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540d6c9-12ab-466b-9613-67d00c2d7f9f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.058315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540d6c9-12ab-466b-9613-67d00c2d7f9f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.058636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540d6c9-12ab-466b-9613-67d00c2d7f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.104726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43302b1-9f51-46cb-9ed3-16118915714a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:31.127803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c540d6c9-12ab-466b-9613-67d00c2d7f9f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:31.129036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c540d6c9-12ab-466b-9613-67d00c2d7f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:31.219939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:31.220454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.220734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.221006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.222637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:31.230205 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 701/1392] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:31.235827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:31.318214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:31.319903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e1080-edca-4151-9e18-efa33787457e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:31.338212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 692/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:31.600106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.600678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.600788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.602295 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 702/1394] 10.222.0.52 () {54 vars in 962 bytes} [Sat Sep 24 20:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:31.608736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfc6e-f044-49c4-8c2f-3dd45e59b13c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.609241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfc6e-f044-49c4-8c2f-3dd45e59b13c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.609333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfc6e-f044-49c4-8c2f-3dd45e59b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.689885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-575bfc6e-f044-49c4-8c2f-3dd45e59b13c None None] MFA Rules not processed for user `99f0d8a1aa0d4bacbd4e4fb2e7864cee`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:31.691196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-575bfc6e-f044-49c4-8c2f-3dd45e59b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:31.771592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 693/1395] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:31.780298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:31.781822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:31.855679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:31.888257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f3ec9-b087-4529-a1ef-02ffaafaa037 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:31.969137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:31.969813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:31.970096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:31.970361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:31.972059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:32.084733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:32.086161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420080aa-c226-4c37-8875-791aeb39398e service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:32.101980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:32.105489 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 703/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:32.697283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a16-ff02-496e-b015-ce2dfd389cb2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:32.697781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a16-ff02-496e-b015-ce2dfd389cb2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:32.697978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9a16-ff02-496e-b015-ce2dfd389cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:32.703418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:32.767390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e7ee9a16-ff02-496e-b015-ce2dfd389cb2 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:32.768573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e7ee9a16-ff02-496e-b015-ce2dfd389cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:32.858667 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 694/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:32.904528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:32.905271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:32.905387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:32.907087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 704/1398] 10.222.0.52 () {54 vars in 943 bytes} [Sat Sep 24 20:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:32.913624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83b3e-c45f-4cca-bd9f-3055f6eeea5d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:32.913983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83b3e-c45f-4cca-bd9f-3055f6eeea5d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:32.914353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83b3e-c45f-4cca-bd9f-3055f6eeea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:32.996636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-46e83b3e-c45f-4cca-bd9f-3055f6eeea5d None None] MFA Rules not processed for user `0b5a7880d6ec47f381af9ef1f09a767c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:32.998200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-46e83b3e-c45f-4cca-bd9f-3055f6eeea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:33.033440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:33.104905 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 695/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Sat Sep 24 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:33.115384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:33.116949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:33.259739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aa175-f51f-4a79-9477-03486b613ac1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:33.359604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:33.379153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:33.379774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:33.380050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:33.380319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:33.382069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:33.471377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:33.472973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b874a-4f99-430b-a0b7-2b35d22c32af service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:33.481184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 705/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:40:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:40:33.837170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:33.840211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:33.840211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:33.841709 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 696/1401] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:33.847009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939586ad-94ad-40ad-986c-c85816b2a408 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:33.847292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939586ad-94ad-40ad-986c-c85816b2a408 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:33.847565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939586ad-94ad-40ad-986c-c85816b2a408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:33.918181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-939586ad-94ad-40ad-986c-c85816b2a408 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:33.920100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-939586ad-94ad-40ad-986c-c85816b2a408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:34.000961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:34.036161 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 706/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:34.385129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:34.386821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:34.557441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aede9cb-9589-4aaf-b288-3cc1b4e42aa0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:34.566464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:34.691293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:34.691913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:34.692184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:34.692457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:34.694072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:34.802623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:34.804833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781b14f-7593-41f6-bf25-d90e29b53fbb service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:34.827029 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 697/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:37.380545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:37.381207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:37.381207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:37.386906 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 707/1404] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:37.395265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090166-db78-4fd5-b478-fcf9b161b784 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:37.395905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090166-db78-4fd5-b478-fcf9b161b784 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:37.395905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090166-db78-4fd5-b478-fcf9b161b784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:37.420362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:37.477024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a090166-db78-4fd5-b478-fcf9b161b784 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:37.478363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a090166-db78-4fd5-b478-fcf9b161b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:37.586414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 698/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:37.965149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:37.967634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:38.124041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945bc9-b018-41a3-88f3-1585a3405ffd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:38.183784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:38.248820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:38.249431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:38.249813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:38.250038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:38.252085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:38.357289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:38.359101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69369f9-6810-497f-9b9f-b169a6657abd service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:38.379153 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 708/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:40.865030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:40.865650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:40.865650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:40.866942 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 699/1407] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:40.872422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9368-33b0-4f21-925c-3f05a6f7475e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:40.872906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9368-33b0-4f21-925c-3f05a6f7475e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:40.873105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9368-33b0-4f21-925c-3f05a6f7475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:40.948958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a0ae9368-33b0-4f21-925c-3f05a6f7475e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:40.951002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0ae9368-33b0-4f21-925c-3f05a6f7475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:40.967347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:41.058480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 709/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:40:41.411100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:40:41.412643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:41.509603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:41.575702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa434d9-e3f0-4b0e-8627-1507269b8d68 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:40:41.711396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:40:41.712178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:41.712459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:41.712731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:41.714416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:41.795537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:41.844821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:40:41.846454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f0ef8-6bbb-4c8c-b3cc-0aff81b009ad service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:40:41.870549 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 700/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:40:51.056963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=37 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:51.867945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=22 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:57.708136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abf365-c5d0-4a4e-8ed8-9422eaa27e09 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:57.708771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abf365-c5d0-4a4e-8ed8-9422eaa27e09 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:57.708771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abf365-c5d0-4a4e-8ed8-9422eaa27e09 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:57.710058 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 710/1410] 10.222.0.52 () {54 vars in 918 bytes} [Sat Sep 24 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:57.716015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aacb2-28b9-447f-9689-0c9747220123 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:57.716525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aacb2-28b9-447f-9689-0c9747220123 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:57.716643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239aacb2-28b9-447f-9689-0c9747220123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:57.791561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-239aacb2-28b9-447f-9689-0c9747220123 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:57.792750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-239aacb2-28b9-447f-9689-0c9747220123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:57.899342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 701/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Sat Sep 24 20:40:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:40:57.914284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015770b-32de-4e87-b607-3d549848c8ec None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:57.914591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015770b-32de-4e87-b607-3d549848c8ec None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:57.914888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015770b-32de-4e87-b607-3d549848c8ec None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:57.916234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 711/1412] 10.222.0.52 () {54 vars in 918 bytes} [Sat Sep 24 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:40:57.921452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b54a3-d8f5-4f83-8d75-586bcf00d77e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:40:57.921738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b54a3-d8f5-4f83-8d75-586bcf00d77e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:40:57.922012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b54a3-d8f5-4f83-8d75-586bcf00d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:40:57.993917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:40:57.997246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-570b54a3-d8f5-4f83-8d75-586bcf00d77e None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:40:57.999096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-570b54a3-d8f5-4f83-8d75-586bcf00d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:40:58.114129 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 702/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Sat Sep 24 20:40:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:00.969683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:00.970218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:00.970218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:00.971501 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:00.976867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddab6-5370-4584-9327-46eff9957654 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:00.977301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddab6-5370-4584-9327-46eff9957654 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:00.977408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddab6-5370-4584-9327-46eff9957654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:01.045514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-945ddab6-5370-4584-9327-46eff9957654 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:01.046731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-945ddab6-5370-4584-9327-46eff9957654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:01.100823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:01.143841 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:01.494130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:01.495678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:01.641738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1d0b-a662-405f-8517-0327fc59ba1e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:01.713470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:01.754924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:01.755525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:01.755797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:01.756063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:01.757633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:01.846270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:01.847890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa254e-bf57-4252-bc67-aa7094af5b53 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:01.865973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 713/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:03.615458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:03.616014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:03.616014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:03.617313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:03.622884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bbda-eb4b-404a-bcd5-74f43fba25b9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:03.623316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bbda-eb4b-404a-bcd5-74f43fba25b9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:03.623432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bbda-eb4b-404a-bcd5-74f43fba25b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:03.699299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5099bbda-eb4b-404a-bcd5-74f43fba25b9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:03.700498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5099bbda-eb4b-404a-bcd5-74f43fba25b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:03.726912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:03.798697 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:04.016359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:04.017839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:04.163660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e7c7-6b10-41d0-9282-c7193d5c2469 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:04.203864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:04.280023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:04.280255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:04.280666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:04.280937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:04.282537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:04.370332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:04.373162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c2462-4fa0-4c29-85fe-9a441d406fba service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:04.390799 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 705/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:06.115632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:06.116174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:06.116292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:06.117622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:06.123256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3fa59-b170-4af8-9d91-cf7fbb38c1fc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:06.123701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3fa59-b170-4af8-9d91-cf7fbb38c1fc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:06.123805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3fa59-b170-4af8-9d91-cf7fbb38c1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:06.177732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:06.200351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-50a3fa59-b170-4af8-9d91-cf7fbb38c1fc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:06.201549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-50a3fa59-b170-4af8-9d91-cf7fbb38c1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:06.303301 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:06.467726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:06.469176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:06.617298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e733-767b-4775-bb36-33c5a8bd4858 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:06.629762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:06.738674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:06.739201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:06.739478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:06.739742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:06.741346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:06.835700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:06.838056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbce6ac-23b4-433a-bb1e-d869c9dcff61 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:06.856668 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 716/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:08.541698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:08.542290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:08.542290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:08.544248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:08.549122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c227df-8e29-4a11-adac-b1e90cb6fd04 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:08.549562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c227df-8e29-4a11-adac-b1e90cb6fd04 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:08.549691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c227df-8e29-4a11-adac-b1e90cb6fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:08.569755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:08.626023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-05c227df-8e29-4a11-adac-b1e90cb6fd04 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:08.627270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05c227df-8e29-4a11-adac-b1e90cb6fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:08.723984 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:08.883327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:08.884899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:08.995438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:09.031338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba805418-56e5-426e-8210-860fda8689d5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:09.144060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:09.144621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:09.145310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:09.145310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:09.146722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:09.222747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:09.236609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:09.238237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1eeda4-8d26-4296-80fa-3d6c30351c35 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:09.257218 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 708/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:10.931199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:10.931737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:10.931737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:10.932998 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:10.938914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd94fc3-b375-43f0-b506-20edb55520d4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:10.939362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd94fc3-b375-43f0-b506-20edb55520d4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:10.939469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd94fc3-b375-43f0-b506-20edb55520d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:11.014614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1dd94fc3-b375-43f0-b506-20edb55520d4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:11.015819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1dd94fc3-b375-43f0-b506-20edb55520d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:11.111214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:11.273722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:11.275238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:11.359749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:11.422420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1beb4-1fd4-49d0-95b0-29c0f7a47a97 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:11.537235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:11.537755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:11.538026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:11.538290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:11.539898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:11.587869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:11.630080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:11.631721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62a4ef-a688-4fb5-a59c-25fbf8281789 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:11.650667 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 719/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:13.274520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:13.275108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:13.275108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:13.276424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:13.281749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2deef-c88c-4926-8702-07f6cdee0fda None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:13.282186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2deef-c88c-4926-8702-07f6cdee0fda None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:13.282282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2deef-c88c-4926-8702-07f6cdee0fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:13.358867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dda2deef-c88c-4926-8702-07f6cdee0fda None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:13.359996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dda2deef-c88c-4926-8702-07f6cdee0fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:13.454705 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:13.616332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:13.617770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:13.667471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:13.764279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11051c80-dbed-4662-8e7f-26237d59468e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:13.875696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:13.876215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:13.876493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:13.876763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:13.878361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:13.885212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:13.969656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:13.971298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d166a6-d754-4389-aaf3-0bc055657527 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:13.989054 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 711/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:15.606402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:15.607008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:15.607008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:15.608303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:15.613177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3d2c-9e7d-4bc4-8d5b-6b0e7963763e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:15.613578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3d2c-9e7d-4bc4-8d5b-6b0e7963763e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:15.613750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac3d2c-9e7d-4bc4-8d5b-6b0e7963763e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:15.692109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-51ac3d2c-9e7d-4bc4-8d5b-6b0e7963763e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:15.693312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-51ac3d2c-9e7d-4bc4-8d5b-6b0e7963763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:15.760162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:15.791336 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:15.957579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:15.959023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:15.976333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:16.104173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fbfd1-e1d8-4964-ab8f-fa1ee224e968 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:16.191620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:16.216675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:16.217209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:16.217484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:16.217747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:16.219434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:16.307389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:16.309549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2926f1-573d-4d4c-bc36-82dde97115e9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:16.327597 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 722/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:18.130624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:18.131160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:18.131160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:18.132477 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:18.138086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba9756-d6f7-4345-8454-07e765e9b417 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:18.138578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba9756-d6f7-4345-8454-07e765e9b417 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:18.138679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba9756-d6f7-4345-8454-07e765e9b417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:18.214822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-72ba9756-d6f7-4345-8454-07e765e9b417 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:18.216071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-72ba9756-d6f7-4345-8454-07e765e9b417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:18.257362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:18.312502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:18.474472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:18.475961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:18.619580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261dd5a-1c31-4e98-ae47-4b104285fd08 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:18.674898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:18.734513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:18.735045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:18.735325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:18.735592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:18.737166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:18.826327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:18.827982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c2b2c-8f7f-4bf6-be4d-c8ae1832803b service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:18.849709 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 714/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:20.469811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:20.470358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:20.470358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:20.471752 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:20.476561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1968dc-e42c-42be-81ad-d24ea42fc351 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:20.476960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1968dc-e42c-42be-81ad-d24ea42fc351 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:20.477107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1968dc-e42c-42be-81ad-d24ea42fc351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:20.552946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5d1968dc-e42c-42be-81ad-d24ea42fc351 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:20.554149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5d1968dc-e42c-42be-81ad-d24ea42fc351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:20.556233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:20.652939 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:20.817065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:20.818604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:20.963127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1055d-ba0b-4026-b3a6-85a34597137a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:20.990273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:21.075964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:21.076484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:21.076756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:21.077018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:21.078623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:21.167092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:21.169357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c7372-cfe1-4f99-9015-eeadeeb52dea service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:21.188140 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 725/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:23.525294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:23.525854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:23.525854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:23.527719 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:23.533436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043dce2-cec6-40fa-9027-10508d0d6db5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:23.533865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043dce2-cec6-40fa-9027-10508d0d6db5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:23.534001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043dce2-cec6-40fa-9027-10508d0d6db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:23.577851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:23.604039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f043dce2-cec6-40fa-9027-10508d0d6db5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:23.605275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f043dce2-cec6-40fa-9027-10508d0d6db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:23.702313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:24.044595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:24.046157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:24.149767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514162c1-f66a-4205-8074-bc10d89543ed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:24.182660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:24.224782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:24.225380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:24.225654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:24.225919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:24.227538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:24.338776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:24.340794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02273b25-c3fe-4f2a-b40f-6a53a14e489c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:24.358784 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 717/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:27.056835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:27.057413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:27.057413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:27.059380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:27.064617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61793e25-4a54-4e2c-b47e-631571304de9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:27.065038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61793e25-4a54-4e2c-b47e-631571304de9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:27.065153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61793e25-4a54-4e2c-b47e-631571304de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:27.131026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-61793e25-4a54-4e2c-b47e-631571304de9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:27.132215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61793e25-4a54-4e2c-b47e-631571304de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:27.145554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:27.229980 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:27.577931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:27.579559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:27.682474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034c70d-3435-4685-9245-c541c02f7f7e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:27.695457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:27.757811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:27.758413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:27.758721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:27.758990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:27.760578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:27.869390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:27.871671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8936f1c-8ac9-4a73-8ea6-fc5acf8916da service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:27.890294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 728/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:30.641768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:30.642236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:30.642346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:30.643681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:30.649044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bbd46-ca8a-486a-9782-975d992c01ae None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:30.649461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bbd46-ca8a-486a-9782-975d992c01ae None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:30.649574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bbd46-ca8a-486a-9782-975d992c01ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:30.702526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:30.722443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-532bbd46-ca8a-486a-9782-975d992c01ae None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:30.723678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-532bbd46-ca8a-486a-9782-975d992c01ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:30.829529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:31.170958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:31.172915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:31.276368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4153a7a-a495-4899-9572-c7d69e8b7eb7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:31.318869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:31.353834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:31.354437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:31.354747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:31.355053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:31.356705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:31.467243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:31.468706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52812ac8-d9fd-4f81-b304-f4f93a8f72b0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:31.487192 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 720/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:34.051937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:34.052459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:34.052459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:34.053735 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:34.059352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815adc-f24a-4981-b7bb-3cd0bb87f731 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:34.059760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815adc-f24a-4981-b7bb-3cd0bb87f731 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:34.059934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c815adc-f24a-4981-b7bb-3cd0bb87f731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:34.125306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4c815adc-f24a-4981-b7bb-3cd0bb87f731 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:34.126510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c815adc-f24a-4981-b7bb-3cd0bb87f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:34.146936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:34.222806 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:34.593208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:34.594774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:34.700195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499dcca-74a9-490e-908d-93d6412656c2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:34.721109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:34.774190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:34.774825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:34.775097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:34.775381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:34.777280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:34.885026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:34.887113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fc66c-faf3-42c2-9a94-37e8ffa5c069 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:34.905334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 731/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:37.442409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:37.442971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:37.442971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:37.444264 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:37.450435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c2-fc3d-4d65-a3f4-2dd9e45adbd2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:37.450870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c2-fc3d-4d65-a3f4-2dd9e45adbd2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:37.451013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29f7c2-fc3d-4d65-a3f4-2dd9e45adbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:37.514371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:37.517390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6a29f7c2-fc3d-4d65-a3f4-2dd9e45adbd2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:37.518940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6a29f7c2-fc3d-4d65-a3f4-2dd9e45adbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:37.615725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:37.965531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:37.967102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:38.070699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaceb7-554f-45a6-8379-0d660d112c97 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:38.120256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:38.145251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:38.145851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:38.146337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:38.146453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:38.148035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:38.259512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:38.261677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09a337-620c-455f-9ca8-a9ecb1a48ca1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:38.281881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 723/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:40.761892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:40.762522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:40.762522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:40.763788 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:40.769099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb71fe1-871f-44d4-a69c-0d48d42bb01a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:40.769512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb71fe1-871f-44d4-a69c-0d48d42bb01a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:40.769646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb71fe1-871f-44d4-a69c-0d48d42bb01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:40.837126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bcb71fe1-871f-44d4-a69c-0d48d42bb01a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:40.838321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bcb71fe1-871f-44d4-a69c-0d48d42bb01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:40.866865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:40.933514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:41.277027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:41.278702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:41.380804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094de0b-cb27-4394-a5c9-df8db9777853 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:41.409199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:41.458534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:41.459143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:41.459418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:41.459686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:41.461278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:41.568276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:41.569657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee7c65-38c2-4781-b67d-efcc6a9bab61 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:41.588380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 734/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:44.062679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:44.063207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:44.063207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:44.064468 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:44.070033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c92d-d53e-4864-b9ea-8026cce77660 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:44.070485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c92d-d53e-4864-b9ea-8026cce77660 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:44.070627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c92d-d53e-4864-b9ea-8026cce77660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:44.135414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4957c92d-d53e-4864-b9ea-8026cce77660 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:44.136697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4957c92d-d53e-4864-b9ea-8026cce77660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:44.144836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:44.234507 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:44.577227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:44.578815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:44.682945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b02bd-45cb-48aa-8ce4-d9f9bec81c5a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:44.739438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:44.757753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:44.758353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:44.758663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:44.758934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:44.760513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:44.869469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:44.871084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62c073-d560-499e-9610-f1b5071d9e8e service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:44.889245 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 726/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:47.359434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:47.359967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:47.360057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:47.361344 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:47.366646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab5-a974-41bc-82ab-c572085aa07b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:47.367064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab5-a974-41bc-82ab-c572085aa07b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:47.367178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766bab5-a974-41bc-82ab-c572085aa07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:47.435295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f766bab5-a974-41bc-82ab-c572085aa07b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:47.436501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f766bab5-a974-41bc-82ab-c572085aa07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:47.474156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:47.532866 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:47.882069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:47.883662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:47.987284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644e883-b866-46d0-a5f2-9386bb548069 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:48.024049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:48.060777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:48.061377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:48.061651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:48.061916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:48.063525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:48.171965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:48.173381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7d9fc-732b-4db9-9ccf-eec4107869a4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:48.191022 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 737/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:50.634472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:50.635548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:50.635753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:50.637479 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:50.642973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf2f-26bc-4d20-8efd-3c716ef8f38a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:50.643390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf2f-26bc-4d20-8efd-3c716ef8f38a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:50.643494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfaf2f-26bc-4d20-8efd-3c716ef8f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:50.708708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-95cfaf2f-26bc-4d20-8efd-3c716ef8f38a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:50.709897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95cfaf2f-26bc-4d20-8efd-3c716ef8f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:50.728467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:50.808962 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:51.159647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:51.161199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:51.262990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9fb7c-66e3-49c5-a004-5bc119629b63 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:51.331930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:51.339745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:51.340345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:51.340622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:51.340893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:51.342514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:51.456346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:51.457831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731d97c-9b7f-4d66-a610-c95f102bf547 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:51.477117 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 729/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:41:53.904466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:53.904990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:53.904990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:53.906530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:41:53.911943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074eb39-a825-4a2f-9285-04ca1597969e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:53.912373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074eb39-a825-4a2f-9285-04ca1597969e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:53.912478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074eb39-a825-4a2f-9285-04ca1597969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:53.978369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d074eb39-a825-4a2f-9285-04ca1597969e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:41:53.979653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d074eb39-a825-4a2f-9285-04ca1597969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:54.027328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:54.076547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:41:54.418830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:41:54.420362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:54.529038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e8cd0-df20-4762-8880-323063d5a7fc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:41:54.576664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:41:54.603117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:41:54.603719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:41:54.603997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:41:54.604798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:41:54.606384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:41:54.717808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:41:54.719194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f645db-5977-4931-b676-488ecf2d0ecc service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:41:54.737424 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 740/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:04.075134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=22 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:04.735241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=66 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:19.451439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc55c1-d19d-4628-a5d6-a2ea5e0f46f6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:19.452045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc55c1-d19d-4628-a5d6-a2ea5e0f46f6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:19.452045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc55c1-d19d-4628-a5d6-a2ea5e0f46f6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:19.453327 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 731/1471] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:19.458836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743c77-d184-49ca-b340-c2c3e2e81cfb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:19.459294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743c77-d184-49ca-b340-c2c3e2e81cfb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:19.459426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef743c77-d184-49ca-b340-c2c3e2e81cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:19.527678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ef743c77-d184-49ca-b340-c2c3e2e81cfb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:19.528885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ef743c77-d184-49ca-b340-c2c3e2e81cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:19.624645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 741/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:42:19.976597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:19.976878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:19.977145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:19.979256 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 732/1473] 10.222.0.52 () {54 vars in 962 bytes} [Sat Sep 24 20:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:19.984358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a27ea-75de-4bee-9c4f-f400d1fd20d9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:19.984643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a27ea-75de-4bee-9c4f-f400d1fd20d9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:19.984913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a27ea-75de-4bee-9c4f-f400d1fd20d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:20.062354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a07a27ea-75de-4bee-9c4f-f400d1fd20d9 None None] MFA Rules not processed for user `99f0d8a1aa0d4bacbd4e4fb2e7864cee`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:20.063597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a07a27ea-75de-4bee-9c4f-f400d1fd20d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:20.084119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:20.139511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 742/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 20:42:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:42:20.147384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:20.148920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:20.251235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acfb18-5c72-45d0-b281-d385f3ece2e0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:20.325956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:20.326587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:20.326864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:20.327128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:20.328693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:20.383624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:20.440219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:20.441661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60365032-2811-4c29-b44c-8859480d4c33 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:20.460649 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 733/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:22.978260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:22.978836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:22.979054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:22.980409 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 743/1476] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:42:22.985520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a22cb-3e4b-4f81-877b-22209b81ec66 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:22.985949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a22cb-3e4b-4f81-877b-22209b81ec66 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:22.986070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a22cb-3e4b-4f81-877b-22209b81ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:23.054191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7c2a22cb-3e4b-4f81-877b-22209b81ec66 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:23.055447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7c2a22cb-3e4b-4f81-877b-22209b81ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:23.152580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 734/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:42:23.479927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:23.481490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:23.605751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50034e1-7e60-472b-8cde-e798bf8c547d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:23.658319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:23.703425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:23.704034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:23.704307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:23.704574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:23.705195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:23.706981 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:42:23.706981 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:42:23.706981 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:42:23.708116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce168d3-815d-4248-ac34-0f83e4196cb8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:23.727345 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 744/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Sat Sep 24 20:42:23 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:42:26.179279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:26.179853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:26.179853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:26.181228 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 735/1479] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:26.186996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ddb7-f777-4a20-b5c6-8ae2ef193072 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:26.187446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ddb7-f777-4a20-b5c6-8ae2ef193072 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:26.187558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ddb7-f777-4a20-b5c6-8ae2ef193072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:26.253068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-08a9ddb7-f777-4a20-b5c6-8ae2ef193072 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:26.254442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08a9ddb7-f777-4a20-b5c6-8ae2ef193072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:26.352174 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 745/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:42:26.707233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:26.708738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:26.719806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:26.866759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36678b4a-30a6-436a-bdec-72995543ab7c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:26.946670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:26.980281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:26.980871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:26.981144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:26.981417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:26.983030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:27.198625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:27.200159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe51fa-b7b3-493b-ae08-2eabb8f8ee24 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:27.218204 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 736/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:29.831412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:29.832003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:29.832003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:29.833317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 746/1482] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:29.840191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebbd65-0667-460b-8b0d-365879be1836 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:29.840700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebbd65-0667-460b-8b0d-365879be1836 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:29.840888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebbd65-0667-460b-8b0d-365879be1836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:29.907831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3cebbd65-0667-460b-8b0d-365879be1836 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:29.908948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3cebbd65-0667-460b-8b0d-365879be1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:29.954314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:30.003498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 737/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:42:30.355578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:30.357112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:30.380367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:30.459067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b407d-146f-4b0a-a6d6-22a7e555c7b5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:30.533350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:30.533952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:30.534230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:30.534526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:30.536116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:30.610532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:30.645880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:30.647367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb3c30-873f-413a-887d-04ec222cf078 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:30.666416 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 747/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:33.141589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:33.142126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:33.142126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:33.143822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 738/1485] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:33.149190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26c98-d191-450e-b05d-d2b4bfe854dd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:33.149603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26c98-d191-450e-b05d-d2b4bfe854dd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:33.149776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26c98-d191-450e-b05d-d2b4bfe854dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:33.216242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5fe26c98-d191-450e-b05d-d2b4bfe854dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:33.217454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5fe26c98-d191-450e-b05d-d2b4bfe854dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:33.313426 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 748/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:42:33.655280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:33.657063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:33.758897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba4165-0fe3-406e-bc7a-7cf92ece5bec None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:33.830887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:33.832459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:33.833109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:33.833394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:33.833846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:33.835777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:33.943955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:33.945322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9439c-e0ef-4ca2-9373-b32271a5310d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:33.962774 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 739/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:36.699960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:36.700469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:36.700586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:36.701846 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 749/1488] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:36.707037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88360-e29f-4eea-8fb6-8b64731dd927 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:36.707497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88360-e29f-4eea-8fb6-8b64731dd927 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:36.707591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a88360-e29f-4eea-8fb6-8b64731dd927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:36.772308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f9a88360-e29f-4eea-8fb6-8b64731dd927 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:36.773516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9a88360-e29f-4eea-8fb6-8b64731dd927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:36.825156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:36.868571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 740/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:42:37.219352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:37.220930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:37.258603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:37.326670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d0692-70c2-4a25-a6d8-7c22b5367f53 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:37.401354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:37.401863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:37.402230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:37.402524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:37.405070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:37.498337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:37.534290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:37.535776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054842-7d84-4bd2-b155-8a7e1cea7e44 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:37.559601 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 750/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:40.312060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:40.312717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:40.312717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:40.313937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 741/1491] 10.222.0.52 () {54 vars in 927 bytes} [Sat Sep 24 20:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:42:40.319238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460352-328a-4ab6-878f-17e4205c3d06 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:40.319695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460352-328a-4ab6-878f-17e4205c3d06 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:40.319804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b460352-328a-4ab6-878f-17e4205c3d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:40.386844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9b460352-328a-4ab6-878f-17e4205c3d06 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:42:40.388061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b460352-328a-4ab6-878f-17e4205c3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:40.489321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 751/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Sat Sep 24 20:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:42:40.859494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:42:40.861033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:40.963495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7d90f-7e8c-4c05-bf9f-4f9311033986 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:42:41.035482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:42:41.036075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:42:41.036348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:42:41.036610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:42:41.038174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:42:41.047324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:41.147324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:42:41.149078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce9293-31be-458e-9e6f-4e26401193a4 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:42:41.169541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 742/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:42:50.487827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=82 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:42:51.167129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=50 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:43:57.397292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.397793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.397902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.480682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:43:57.481804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:57.582380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 752/1494] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:43:57.591267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.591749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.591856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.612107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:43:57.613283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:57.622078 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 743/1495] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:43:57.626612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.638687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821d15e-9376-4dcb-9c2a-c36459c2a49e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.643486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.644084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.644358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.644628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.645265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:43:57.645562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.646661 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.646661 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:43:57.646661 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:43:57.647544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.660048 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 753/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:43:57.668406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.677306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef32364-ba83-464e-b29a-8964244cc45b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.681964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.682587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.682861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.683127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.683745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.685492 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.685492 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:43:57.685492 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:43:57.686715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.722830 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 744/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:43:57 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:43:57.727116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.737159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38ec06-706b-4d45-83cd-322261ef127e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.785950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:43:57.793238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.793867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.794143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.794411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.795161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.796203 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.796203 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:43:57.796203 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:43:57.797410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.818396 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 754/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:43:57 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:43:57.822419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.832390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c08d5-97bd-48c2-9d4e-c4d83ab6bca8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.838002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.838512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.838802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.839074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.839708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.840719 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.840719 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:43:57.840719 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:43:57.841600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.854682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 745/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:43:57.858520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.868467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b052667-0a5b-43dc-b32b-095ece3a39f6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.873335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.874133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.874545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.874932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.875851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.877348 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.877348 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:43:57.877348 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:43:57.878707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415853d-b32d-47bb-94a2-1e7ff7e93813 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.894873 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 755/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:43:57.898740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:57.909711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1dda2-eee0-41e1-ace6-02d17a8e6560 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:57.914289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:57.914972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.915293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.915736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] PATH_INFO: `/v3/projects/7aed35640ce042a59219f4601d6f831b/users/1a45c96394e443688981a046f4a95ae2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.929922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aed35640ce042a59219f4601d6f831b, user_id=1a45c96394e443688981a046f4a95ae2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:57.931169 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:57.931169 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:43:57.931169 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:43:57.933087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2fdba-f60b-4579-9f38-d46170011efd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:57.945789 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 746/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:43:57 2022] PUT /identity/v3/projects/7aed35640ce042a59219f4601d6f831b/users/1a45c96394e443688981a046f4a95ae2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:43:57.958469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7ddc-cb6d-4653-a535-70ddc76ac576 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:57.958784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7ddc-cb6d-4653-a535-70ddc76ac576 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:57.959075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd7ddc-cb6d-4653-a535-70ddc76ac576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:57.986554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d4dd7ddc-cb6d-4653-a535-70ddc76ac576 None None] MFA Rules not processed for user `1a45c96394e443688981a046f4a95ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:43:57.987816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4dd7ddc-cb6d-4653-a535-70ddc76ac576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:58.050579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 756/1502] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:43:58.064016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa220bf-13e4-40cf-b53d-ef2815ee5360 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.064297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa220bf-13e4-40cf-b53d-ef2815ee5360 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.064591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa220bf-13e4-40cf-b53d-ef2815ee5360 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.065902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 747/1503] 10.222.0.52 () {54 vars in 963 bytes} [Sat Sep 24 20:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:43:58.070581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdc566-c7da-4b84-910a-b88a896ece84 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.070871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdc566-c7da-4b84-910a-b88a896ece84 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.071148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fdc566-c7da-4b84-910a-b88a896ece84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.131573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b9fdc566-c7da-4b84-910a-b88a896ece84 None None] MFA Rules not processed for user `c66ad10d7496429da3b81648cf9e83f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:43:58.132784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b9fdc566-c7da-4b84-910a-b88a896ece84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:58.208901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 757/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Sat Sep 24 20:43:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:43:58.216843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa220bf-13e4-40cf-b53d-ef2815ee5360 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:58.228260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa220bf-13e4-40cf-b53d-ef2815ee5360 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:58.231892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:58.232506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.232865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.233116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.241882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:58.243794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:58.252674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 748/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:43:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:43:58.300074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8857d-d7df-4429-9bb7-e2305a9bbec2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.300365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8857d-d7df-4429-9bb7-e2305a9bbec2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.300645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8857d-d7df-4429-9bb7-e2305a9bbec2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.301906 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 758/1506] 10.222.0.52 () {52 vars in 830 bytes} [Sat Sep 24 20:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:43:58.305500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:58.367323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd97b10-a56c-4185-b57e-3a37ff85f79c service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:58.372809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:58.373431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.373709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.373983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] PATH_INFO: `/v3/users/1a45c96394e443688981a046f4a95ae2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.375164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a45c96394e443688981a046f4a95ae2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:58.376184 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:58.376184 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:43:58.376184 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:43:58.377390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:58.445347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a45c96394e443688981a046f4a95ae2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:58.445998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a45c96394e443688981a046f4a95ae2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:58.455305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a45c96394e443688981a046f4a95ae2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:58.469253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a45c96394e443688981a046f4a95ae2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:58.469984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a45c96394e443688981a046f4a95ae2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:58.492107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:43:58.501542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 749/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:43:58 2022] DELETE /identity/v3/users/1a45c96394e443688981a046f4a95ae2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:43:58.506080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.506591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.506775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.527138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:43:58.528343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:58.537504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:43:58.586863 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 759/1508] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:43:58.596515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:58.598025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:58.678457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcda327-8cd2-4b0c-8b56-9c7cd9ff11ea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:58.683050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:58.683649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:58.683925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:58.684190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:58.692931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:58.694393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3c11e-776e-45a0-a921-04676bbaf963 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:58.704131 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 750/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:43:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:43:59.027916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:43:59.029377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:43:59.041559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36770d50-3caa-4ae0-977c-b0acfa63fe0a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:43:59.047213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:43:59.047888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:43:59.048199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:43:59.048527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] PATH_INFO: `/v3/projects/7aed35640ce042a59219f4601d6f831b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:43:59.049724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aed35640ce042a59219f4601d6f831b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:43:59.050785 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:43:59.050785 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:43:59.050785 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:43:59.052116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:43:59.113788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aed35640ce042a59219f4601d6f831b', 'request_id': 'req-14370517-0091-4c20-a4bd-c59ede9934e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:59.120982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aed35640ce042a59219f4601d6f831b', 'request_id': 'req-14370517-0091-4c20-a4bd-c59ede9934e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:59.138334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aed35640ce042a59219f4601d6f831b', 'request_id': 'req-14370517-0091-4c20-a4bd-c59ede9934e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:59.152550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] The token cache is being invalidate because project 7aed35640ce042a59219f4601d6f831b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:43:59.152864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:59.153483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-14370517-0091-4c20-a4bd-c59ede9934e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:43:59.155097 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 760/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:43:59 2022] DELETE /identity/v3/projects/7aed35640ce042a59219f4601d6f831b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:44:08.699938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:44:09.149012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:47:21.561908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69c96-1e32-45a2-9ada-9bfda05e4a43 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:21.562413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69c96-1e32-45a2-9ada-9bfda05e4a43 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:21.562677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69c96-1e32-45a2-9ada-9bfda05e4a43 None None] PATH_INFO: `/v3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:21.563836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 751/1511] 10.222.0.25 () {54 vars in 925 bytes} [Sat Sep 24 20:47:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:47:21.591084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bda7-12ca-4954-87e9-89d506b8cbe0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:21.591552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bda7-12ca-4954-87e9-89d506b8cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:21.591657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61bda7-12ca-4954-87e9-89d506b8cbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:21.643060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6e61bda7-12ca-4954-87e9-89d506b8cbe0 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:47:21.644301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e61bda7-12ca-4954-87e9-89d506b8cbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:47:21.715590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 761/1512] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 24 20:47:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:47:25.510616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be662cb-a31b-446e-9309-d212b1cf6bf0 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:25.511069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be662cb-a31b-446e-9309-d212b1cf6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:25.511169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be662cb-a31b-446e-9309-d212b1cf6bf0 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:25.512514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 752/1513] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:47:25.517370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8a0a4-93db-485a-b41e-b9ad7a4a2959 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:25.517853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8a0a4-93db-485a-b41e-b9ad7a4a2959 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:25.518006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8a0a4-93db-485a-b41e-b9ad7a4a2959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:25.537888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-61d8a0a4-93db-485a-b41e-b9ad7a4a2959 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:47:25.539143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-61d8a0a4-93db-485a-b41e-b9ad7a4a2959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:47:25.547760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 762/1514] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:47:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:47:25.555427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be662cb-a31b-446e-9309-d212b1cf6bf0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:47:25.569052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be662cb-a31b-446e-9309-d212b1cf6bf0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:47:25.574853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:47:25.575444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:25.575720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:25.575987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:25.584610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:47:25.586222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574ed94-9dbc-4be0-9856-5e2c61a7ea04 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:47:25.596815 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 753/1515] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:47:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:47:34.260868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802860-048a-4ca2-ab40-7a7b63438605 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:34.261475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802860-048a-4ca2-ab40-7a7b63438605 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:34.261475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802860-048a-4ca2-ab40-7a7b63438605 None None] PATH_INFO: `/v3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:34.262696 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 763/1516] 10.222.0.26 () {54 vars in 925 bytes} [Sat Sep 24 20:47:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:47:34.269004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ccef1-b272-4295-b3bd-9a045f6c3d5d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:34.269391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ccef1-b272-4295-b3bd-9a045f6c3d5d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:34.269548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ccef1-b272-4295-b3bd-9a045f6c3d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:34.289770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-339ccef1-b272-4295-b3bd-9a045f6c3d5d None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:47:34.291018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-339ccef1-b272-4295-b3bd-9a045f6c3d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:47:34.300248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 754/1517] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 24 20:47:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:47:35.533268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=47 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:47:43.776275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d6cf7-56f3-45b6-ade1-f5470c2b5047 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:43.776794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d6cf7-56f3-45b6-ade1-f5470c2b5047 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:43.776897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d6cf7-56f3-45b6-ade1-f5470c2b5047 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:43.778580 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 764/1518] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:47:43.783554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245af9-113a-4453-bd08-4cd9399ca3bd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:43.783979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245af9-113a-4453-bd08-4cd9399ca3bd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:43.784082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245af9-113a-4453-bd08-4cd9399ca3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:43.803860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-60245af9-113a-4453-bd08-4cd9399ca3bd None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:47:43.805034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60245af9-113a-4453-bd08-4cd9399ca3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:47:43.814560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 755/1519] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:47:43.822050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d6cf7-56f3-45b6-ade1-f5470c2b5047 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:47:43.835264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d6cf7-56f3-45b6-ade1-f5470c2b5047 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:47:43.840286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:47:43.840868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:47:43.841146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:47:43.841425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:47:43.849820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:47:43.851546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1b961-0d40-4576-af82-0d3d3f295e72 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:47:43.861659 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 765/1520] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:47:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:47:53.798683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=12 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:47:53.857584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:48:00.408269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:00.408809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:00.408809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:00.410121 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 756/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:48:00.415144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbe234-8c37-47d4-a842-75d4e328be21 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:00.415570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbe234-8c37-47d4-a842-75d4e328be21 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:00.415703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbe234-8c37-47d4-a842-75d4e328be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:00.434856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84fbe234-8c37-47d4-a842-75d4e328be21 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:48:00.436582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84fbe234-8c37-47d4-a842-75d4e328be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:00.493308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 766/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:48:00.659709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:48:00.661321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:00.761679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d90f9-37d4-44a0-a4c8-efbc787eb31e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:48:00.768048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:48:00.768643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:00.768923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:00.769197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:00.777175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:48:00.779362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4e220-089e-4cab-8c93-191ab3a7711a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:48:00.788550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 757/1523] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:48:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:48:02.427731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2911-7491-47d9-9eb6-341c877e5d85 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:02.428312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2911-7491-47d9-9eb6-341c877e5d85 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:02.428312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2911-7491-47d9-9eb6-341c877e5d85 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:02.429607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 767/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 20:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:48:02.434519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d39fe-ef42-472d-b843-a46e2a77fa2d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:02.434963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d39fe-ef42-472d-b843-a46e2a77fa2d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:02.435062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d39fe-ef42-472d-b843-a46e2a77fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:02.456272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0a8d39fe-ef42-472d-b843-a46e2a77fa2d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:48:02.457468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a8d39fe-ef42-472d-b843-a46e2a77fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:02.465341 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 758/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 20:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:48:02.629391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2911-7491-47d9-9eb6-341c877e5d85 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:48:02.643815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2911-7491-47d9-9eb6-341c877e5d85 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:48:02.649419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:48:02.650018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:02.650290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:02.650589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:02.659504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:48:02.661083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2005f-effc-4d51-97be-ee4b6289c540 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:48:02.671412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 768/1526] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:48:06.846164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:06.846784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:06.846784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:06.848043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 759/1527] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:48:06.853237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf912-f93a-491f-b5e2-2ca6b06bb5ad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:06.853646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf912-f93a-491f-b5e2-2ca6b06bb5ad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:06.853799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaf912-f93a-491f-b5e2-2ca6b06bb5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:06.874096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6dbaf912-f93a-491f-b5e2-2ca6b06bb5ad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:48:06.875840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6dbaf912-f93a-491f-b5e2-2ca6b06bb5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:06.883648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 769/1528] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:48:07.224926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:48:07.226647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:07.285002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a709c5-33e1-4df3-9d63-4756677939ea None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:48:07.288575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:48:07.289174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:07.289457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:07.289737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:07.299485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:48:07.300922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69487f-5af1-49ce-9efe-8780de44a2cc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:48:07.310271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 760/1529] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:48:09.696458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4cc3-99e0-4f25-a804-1131c2acc222 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:09.696995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4cc3-99e0-4f25-a804-1131c2acc222 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:09.696995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4cc3-99e0-4f25-a804-1131c2acc222 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:09.698445 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 770/1530] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 20:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:48:09.703828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6145-f5a0-40fa-affa-df6edeaded5c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:09.704234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6145-f5a0-40fa-affa-df6edeaded5c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:09.704371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c6145-f5a0-40fa-affa-df6edeaded5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:09.724391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c13c6145-f5a0-40fa-affa-df6edeaded5c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:48:09.725592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c13c6145-f5a0-40fa-affa-df6edeaded5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:48:09.733911 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 761/1531] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 20:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:48:10.077912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a4cc3-99e0-4f25-a804-1131c2acc222 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:48:10.088819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a4cc3-99e0-4f25-a804-1131c2acc222 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:48:10.092370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:48:10.093636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:48:10.093870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:48:10.094141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:48:10.102954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:48:10.104366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f8cd7-6d75-4e9f-88d4-daaa93c61bf4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:48:10.113533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 771/1532] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:48:19.719121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=78 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:48:20.109782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=42 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:00.480844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b777a5-3534-4205-9be1-6f2914a2895d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:00.481375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b777a5-3534-4205-9be1-6f2914a2895d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:00.481375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b777a5-3534-4205-9be1-6f2914a2895d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:00.482744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 762/1533] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:00.487916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d05ca-47b9-4615-983c-c25d07e02006 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:00.488353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d05ca-47b9-4615-983c-c25d07e02006 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:00.488485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d05ca-47b9-4615-983c-c25d07e02006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:00.509606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-499d05ca-47b9-4615-983c-c25d07e02006 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:00.510881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-499d05ca-47b9-4615-983c-c25d07e02006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:00.520367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 772/1534] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:49:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:10.504764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:34.414640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.415216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.415216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.439417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.440665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.455345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 763/1535] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.464843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.465387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.465387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.491714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.493304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.507135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 773/1536] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.511678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:34.528269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f8a0c-7eff-4003-b51b-0176c8961e05 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:34.539204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:34.539882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.540164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.540436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.541082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:34.541380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:34.542632 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:34.542632 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:49:34.542632 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:49:34.543533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f7c8-392e-40ba-b974-4b71bd58399d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:34.564637 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 764/1537] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:34.572911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:34.584604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd226424-4ca9-46ba-ae51-2424675437e3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:34.589434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:34.590047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.590323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.590618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.591252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:34.592263 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:34.592263 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:49:34.592263 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:49:34.593581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4d043-a437-4316-bacd-2b34cdfebe4a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:34.614140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.614446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.614721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.637276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 774/1538] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:34.642582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883b06-273c-4448-b9c9-b4f4c04e476d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.642582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883b06-273c-4448-b9c9-b4f4c04e476d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.642582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa883b06-273c-4448-b9c9-b4f4c04e476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.661750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.663003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.675405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fa883b06-273c-4448-b9c9-b4f4c04e476d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.676563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa883b06-273c-4448-b9c9-b4f4c04e476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.756326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 765/1539] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:49:34.757240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:34.760321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 775/1540] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.761799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa3dc-4eb6-46c0-a3ba-7614b2a6cb27 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.762703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa3dc-4eb6-46c0-a3ba-7614b2a6cb27 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.763218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa3dc-4eb6-46c0-a3ba-7614b2a6cb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.768137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bdbae-d494-4a71-82cc-dde02b3bbd4d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:34.773547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:34.774382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.774694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.774967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.775863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:34.777084 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:34.777084 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:49:34.777084 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:49:34.778310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c7da-6174-4868-9072-147f1538cdac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:34.791507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-df5aa3dc-4eb6-46c0-a3ba-7614b2a6cb27 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.793238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df5aa3dc-4eb6-46c0-a3ba-7614b2a6cb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.803530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 776/1541] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.804145 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 766/1542] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/users => generated 300 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:34.805506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28047-2bf9-4004-9c6a-4cc14a39a7c8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.805794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28047-2bf9-4004-9c6a-4cc14a39a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.805963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8f2b-a608-43d8-998a-efc0bf9edecd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.806056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e28047-2bf9-4004-9c6a-4cc14a39a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.806247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8f2b-a608-43d8-998a-efc0bf9edecd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.806545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c8f2b-a608-43d8-998a-efc0bf9edecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.827363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-67e28047-2bf9-4004-9c6a-4cc14a39a7c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.828603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-67e28047-2bf9-4004-9c6a-4cc14a39a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.834148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fe6c8f2b-a608-43d8-998a-efc0bf9edecd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.835400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe6c8f2b-a608-43d8-998a-efc0bf9edecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.849241 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 777/1543] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.850370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ce89-6ab2-4b1b-bc6a-2d1017f26c3b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.850715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ce89-6ab2-4b1b-bc6a-2d1017f26c3b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.850956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ce89-6ab2-4b1b-bc6a-2d1017f26c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.861918 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 767/1544] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.866740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5085f-1bda-4686-898a-6c55eaf64fc1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.866740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5085f-1bda-4686-898a-6c55eaf64fc1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.866740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5085f-1bda-4686-898a-6c55eaf64fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.873153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ae30ce89-6ab2-4b1b-bc6a-2d1017f26c3b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.874349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae30ce89-6ab2-4b1b-bc6a-2d1017f26c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.883206 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 778/1545] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.884241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.884523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.884793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.885139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-70f5085f-1bda-4686-898a-6c55eaf64fc1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.886327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-70f5085f-1bda-4686-898a-6c55eaf64fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.895954 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 768/1546] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.897038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ecdb-374a-4767-857d-e089e810a0ac None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.897320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ecdb-374a-4767-857d-e089e810a0ac None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.897774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ecdb-374a-4767-857d-e089e810a0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.906186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.908292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.920789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b9a8ecdb-374a-4767-857d-e089e810a0ac None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.924414 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 779/1547] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.924544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9a8ecdb-374a-4767-857d-e089e810a0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.924638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:34.931938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 769/1548] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.933157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334bf13-ca2e-42af-90fa-9ae946146608 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.933453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334bf13-ca2e-42af-90fa-9ae946146608 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.933713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334bf13-ca2e-42af-90fa-9ae946146608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.933893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e10172-447d-470b-9f0a-03fdd320e79c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:34.939408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:34.940000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.940411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.940895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.941533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:34.943141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c155f-63e4-4bcb-98d2-c1a71bdab6a7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:34.954535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 780/1549] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:34.955535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23eaa4-46ba-406a-9c96-8f0718f555bf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.955816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23eaa4-46ba-406a-9c96-8f0718f555bf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.956083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23eaa4-46ba-406a-9c96-8f0718f555bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.961561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8334bf13-ca2e-42af-90fa-9ae946146608 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.962798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8334bf13-ca2e-42af-90fa-9ae946146608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.972448 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 770/1550] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.973352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171db18-9ab2-4044-9383-a83936476905 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.973599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171db18-9ab2-4044-9383-a83936476905 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.973872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171db18-9ab2-4044-9383-a83936476905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.975700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0f23eaa4-46ba-406a-9c96-8f0718f555bf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.976890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f23eaa4-46ba-406a-9c96-8f0718f555bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:34.984719 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 781/1551] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:34.986852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:34.987142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:34.987415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:34.994850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e171db18-9ab2-4044-9383-a83936476905 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:34.996870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e171db18-9ab2-4044-9383-a83936476905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:35.006249 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 771/1552] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:35.007452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:35.008639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:35.010698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171db18-9ab2-4044-9383-a83936476905 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.016472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 782/1553] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:35.017669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.023336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171db18-9ab2-4044-9383-a83936476905 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.027729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dedc3b-3c74-448a-8346-e1ebfe47001c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.028151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.028850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.029203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.029452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.030098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.030395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.032067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199ef2c-f4b6-4682-95c0-9bcfc453957e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.036257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.036866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.037149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.037426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.038137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.038447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.040101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.044271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 772/1554] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.045506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.045792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.046066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.056666 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 783/1555] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.057467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.073479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:35.074805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:35.075382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbaa7-b05f-4a1f-bb48-355e98a9f41e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.080652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.081250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.081529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.081803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.082473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.084104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.087984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 773/1556] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:35.087984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.096416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5c294-927d-4e0c-865c-520febd80f8e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.100119 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 784/1557] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:35.101016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.101213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.101610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.101936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.102145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.102813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.103387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.104985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.111863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b46779-183d-4c67-8fd4-a9f042aa964e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.116591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.117178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.117457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.117729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.118158 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 774/1558] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.118364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.118741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.118875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.120293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.130132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f85a-92ce-4e40-83ed-577ed2558d04 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.134792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 785/1559] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.134931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.135501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.135611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.135779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.136055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.136832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.137135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.139018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.147976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0f914-097e-45ce-9943-1d36958d3465 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.152859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.153488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.153791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.153923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 775/1560] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.154064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.154725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.163103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.163488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.165011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbc17c-b56c-4161-a494-242405d19f55 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.165171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.170260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.170903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.171181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.171944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.172575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.175505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.180082 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 786/1561] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.180885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.192374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19255cb-dbd8-4fec-a26e-d8fcae3d8e17 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.197343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.198436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.198808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.199086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.199873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.202053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.217769 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 776/1562] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.217769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.236046 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 787/1563] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.237017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.246600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.255213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.296915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1681d-ec50-4737-a6ce-acdb3c9d3748 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.302806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.303449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.303727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.304000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.304640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:35.304943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.306332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d872259-6e19-4498-b0b7-c1ba9866d480 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.306633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.311072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.311721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.311999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.312272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] PATH_INFO: `/v3/projects/6ea3cc3ecfc941758415b8324faf7644/users/7af773989be84d47a75f107f9acc8c69/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.319311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 777/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:35.320100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.330292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d51c79-a9c0-40a3-976b-47cd493793b1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.330735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea3cc3ecfc941758415b8324faf7644, user_id=7af773989be84d47a75f107f9acc8c69, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.331774 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:35.331774 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:49:35.331774 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:49:35.333790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.335149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.335740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.336025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.336299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.337045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.339220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.349412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 788/1565] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:35 2022] PUT /identity/v3/projects/6ea3cc3ecfc941758415b8324faf7644/users/7af773989be84d47a75f107f9acc8c69/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:35.350015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.374063 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 778/1566] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.374063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.425367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38ced0-81e6-49f3-af66-eebd92f9b1a6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.439805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e9fcf-efba-4281-b710-64d7caa5a889 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.444718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.445264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.445543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.445819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.446431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.448448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.477049 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 779/1567] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.477858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.487012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.487638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.488323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.488323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.488798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.490790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.520792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 789/1568] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.521623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.550954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07d8d-7e41-42fb-89c4-077c92cdd7aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.551853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.580796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.591997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b264ed-3089-40e3-ad0f-85ea6a21f440 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.598256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.598479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.598910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.599598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.599960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.602230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.619571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.620201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.620462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.620801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.621521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.623544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.630081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 790/1569] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.630081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.638660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11835f4e-e1fc-48f5-81a6-de50d134562d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.644144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.644817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.645258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.645583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.646434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.648992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa0f19-768a-4d14-96ca-8a6ccf688d6a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.655508 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 780/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.656380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.674939 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 791/1571] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.674939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.674939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.674939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.701354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] MFA Rules not processed for user `7af773989be84d47a75f107f9acc8c69`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:35.702559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:35.722885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440c8-ba79-4b68-8e02-983da23c66e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.727562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.728162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.728439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.728710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.729330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.731304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.758872 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 781/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.759990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.769474 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 792/1573] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:35.770294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.781369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81f4d9-5804-46a7-bcce-ec24639aad71 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.829071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3594cd-f525-4c05-8ae9-5f33dbd75b7d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.835331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.836051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.836325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.836589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.837208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.839935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.840190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.840799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.841076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.841344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.841962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.843926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.863406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 782/1574] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.864476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.869528 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 793/1575] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.870442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.874565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429a9f1-bdd5-4431-b55c-ce29760f51fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.879351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.879932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.880204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.880468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.880630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7c19-e4f4-4926-b748-d0212e79236c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.881082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.883027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.885399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.885982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.886262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.886562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.887200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.888765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.902786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 794/1576] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:35.902786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.905084 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 783/1577] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.906105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.910272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.914330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40276f43-0a25-4c03-9607-aa473f3d32ba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.918592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159594-920e-4046-a115-cfa76c28892d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.920201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.920804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.921086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.921361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.922003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.924217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.924802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.924901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.925086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.925359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.925986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.927613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.929843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:35.943476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 784/1578] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:35.943476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.950784 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 795/1579] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 292 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.950784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.955097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a3f5-7dd7-4b76-9570-5b5bd88b158f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.958456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1929e84-3c3b-4999-aa46-b35acfa894cd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:35.959661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.960385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.960646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.960914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.961532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.963182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:35.963478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.963750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:35.964054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:35.964328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:35.964924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:35.966826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:35.992502 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 785/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/users => generated 286 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.993336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:35.997386 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 796/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:35 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:35.998165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.003365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75402e3-3372-4f7f-a2ad-545f62a5b55a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.060798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.061435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.061714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.061984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.062302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe56f74-d432-46e4-a4bd-fadc27ce59ea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.062643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.064447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.068108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.068986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.069101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.069318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.069953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.071730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.079096 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 786/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.080431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.086389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 797/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.087428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.093755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41b0c0-5faf-4b2f-8ed8-9b4d1e425d74 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.097555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efde3e-06f2-4ff1-b9ca-8556f5f3ba2e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.099233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.099864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.100149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.100421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.101060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.102595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.102721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.103204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.103464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.103739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.104369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.105935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.114159 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 787/1584] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.114914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.119252 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 798/1585] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.119252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.124650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d9764-053d-451a-a646-7545a85744f0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.129391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98d0c8-1df8-4803-b219-0899b9b26b3e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.130148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.131013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.131410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.131688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.132325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.133912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.134391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.135140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.135259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.135523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.136164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.137830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.145082 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 788/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.147565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.148651 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 799/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.149429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.158541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95d583-5a61-41f0-89cd-e29083d5d3d0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.159267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa8a2-fc09-447e-af08-2040c0bbf96a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.163320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.163677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.163909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.164189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.164289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.164463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.164556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.164750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.165094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.165349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.166724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.167300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.176657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 789/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.177453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.188220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d71b9b-f44a-4931-b82d-866adbb59bc4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.192282 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 800/1589] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/users => generated 305 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:36.193171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.196378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.196971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.197271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.197536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.198160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.199791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.202731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63c12-66e7-463c-b509-13cbf726e23f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.208261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.208792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.209069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.209343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.210002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.210636 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 790/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.211459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.211628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.220981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590345b-9235-4b49-9a59-abd1d87d1823 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.225328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 801/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.225328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.228796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.229443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.229716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.229985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] PATH_INFO: `/v3/projects/dcbc4f38ed72401396f32f8ab0261472/users/09dcd838c264480aab44daca185b5769/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.232721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cd02e-7db1-414d-ab80-b8c98b9ac821 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.238256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.238884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.239164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.239434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.240062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.241641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.245458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbc4f38ed72401396f32f8ab0261472, user_id=09dcd838c264480aab44daca185b5769, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.248175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.252592 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 802/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.253633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.263253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 791/1593] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/dcbc4f38ed72401396f32f8ab0261472/users/09dcd838c264480aab44daca185b5769/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.264063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.264871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105159-3dcd-4ffe-b193-b00ac6f91ff5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.328668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.329410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.329685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.330061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] PATH_INFO: `/v3/projects/084b640ae5ed4ce683962557ecdd84ec/users/fb4bd7ce915e4b329d3f8cf1ee5f2d3f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.331219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e71af-c985-4ec0-b667-296670838c81 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.332428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:36.336635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.337261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.337540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.337814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.338671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.340314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.347214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084b640ae5ed4ce683962557ecdd84ec, user_id=fb4bd7ce915e4b329d3f8cf1ee5f2d3f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.349871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.350688 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 792/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.351501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.362166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:36.366291 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 803/1595] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/084b640ae5ed4ce683962557ecdd84ec/users/fb4bd7ce915e4b329d3f8cf1ee5f2d3f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.366785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9c15-24ef-4d49-93ec-d0913476f8a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.367146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.426419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.427054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.427332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.427605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.428234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.429697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e4e7d-c6b3-4ba3-8a64-88694b98f1d3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.429856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.434401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.435028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.435315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.435590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.436388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.438166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.440292 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 793/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.441076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.450536 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 804/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:36.450781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b2b44-3916-4a9b-be85-9c5f117f3391 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.451392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.456634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.457403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.457688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.457965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] PATH_INFO: `/v3/projects/478a31be22cc451b92af447ed8d81242/users/d278977b201c48dda2d73823fbfe4833/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.461661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bcd5e-6b80-49f2-916c-c9546e0493ed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.466163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.466830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.467179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.467445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] PATH_INFO: `/v3/projects/c80ef23e8f684c919bbe01fa65d145af/users/af6db87401034d90bc8836a53414c517/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.473639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=478a31be22cc451b92af447ed8d81242, user_id=d278977b201c48dda2d73823fbfe4833, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.476324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.482774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80ef23e8f684c919bbe01fa65d145af, user_id=af6db87401034d90bc8836a53414c517, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.485778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e4301-7173-4a58-b272-5edb4fe52e6d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.489705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 794/1598] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/478a31be22cc451b92af447ed8d81242/users/d278977b201c48dda2d73823fbfe4833/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.490574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.500546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 805/1599] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/c80ef23e8f684c919bbe01fa65d145af/users/af6db87401034d90bc8836a53414c517/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.501852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.502132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.502406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.533145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] MFA Rules not processed for user `09dcd838c264480aab44daca185b5769`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:36.534089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:36.564628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7fbc1-53fb-4b7a-ad47-7da33c4f11ee admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.597786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 806/1600] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:36.599327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.622040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.622727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.623026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.623301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] PATH_INFO: `/v3/projects/85ca75b9c5844ba28780c8218a113a86/users/08746bd50eda47eaa632fb972b4d2097/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.643400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ca75b9c5844ba28780c8218a113a86, user_id=08746bd50eda47eaa632fb972b4d2097, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.646076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9069720-5461-4a2e-86ad-2f4eab354c1c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.646795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:36.661039 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 795/1601] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/85ca75b9c5844ba28780c8218a113a86/users/08746bd50eda47eaa632fb972b4d2097/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.662262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.662610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.662885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.675001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df58c-bc0c-4fff-8540-fbe93bebff0d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.678627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:36.695449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] MFA Rules not processed for user `fb4bd7ce915e4b329d3f8cf1ee5f2d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:36.696670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:36.730141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.730744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.731020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.731287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] PATH_INFO: `/v3/projects/a9e62e74103b45cc81dfe7900c0baf5a/users/2c01bfb1e8464616aceb3b73c958eb3f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.745740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e62e74103b45cc81dfe7900c0baf5a, user_id=2c01bfb1e8464616aceb3b73c958eb3f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.748456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.761798 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 807/1602] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/a9e62e74103b45cc81dfe7900c0baf5a/users/2c01bfb1e8464616aceb3b73c958eb3f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.762033 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 796/1603] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:36.762858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.767894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.774130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab967-2c53-4286-8928-b0a53aa25cda None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.830801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.831466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.831742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.832012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] PATH_INFO: `/v3/projects/79afe6f2d12f45e994b77b96c44f8239/users/64a47c4d366e4af88ff0386e79158e85/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.835838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8fe6c-b4d7-460c-94be-f6e4281245bd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.841058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.841743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.842136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.842415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] PATH_INFO: `/v3/projects/d29eee0ecb094c65ab13e3d567d3ddf5/users/3a1497a89b1a40acbb321ac88e483117/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.847413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79afe6f2d12f45e994b77b96c44f8239, user_id=64a47c4d366e4af88ff0386e79158e85, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.850156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b2fb5-3d46-41b2-b9d5-87573d9336e6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.858913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29eee0ecb094c65ab13e3d567d3ddf5, user_id=3a1497a89b1a40acbb321ac88e483117, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.862153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d75b4-9163-4007-a886-038216712c81 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:36.864147 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 797/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/79afe6f2d12f45e994b77b96c44f8239/users/64a47c4d366e4af88ff0386e79158e85/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.865370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.865656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.865929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.879693 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 808/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:36 2022] PUT /identity/v3/projects/d29eee0ecb094c65ab13e3d567d3ddf5/users/3a1497a89b1a40acbb321ac88e483117/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:36.881264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682942a-38ba-424d-96e0-00d3c07555a8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.881544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682942a-38ba-424d-96e0-00d3c07555a8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.881814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682942a-38ba-424d-96e0-00d3c07555a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.896639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:36.897861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:36.910395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a682942a-38ba-424d-96e0-00d3c07555a8 None None] MFA Rules not processed for user `af6db87401034d90bc8836a53414c517`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:36.911616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a682942a-38ba-424d-96e0-00d3c07555a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:36.956805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:36.962944 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 798/1606] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:36.964090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:36.974169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d8fa3-a766-4fe0-9980-640513ee026a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:36.978799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:36.979395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.979671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.979940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.980066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 809/1607] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:36.980609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:36.981239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9599-8a93-4087-b319-5f011eec04d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:36.981527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9599-8a93-4087-b319-5f011eec04d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:36.981798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c9599-8a93-4087-b319-5f011eec04d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:36.982454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.005865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:37.010629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-153c9599-8a93-4087-b319-5f011eec04d2 None None] MFA Rules not processed for user `08746bd50eda47eaa632fb972b4d2097`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:37.011927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-153c9599-8a93-4087-b319-5f011eec04d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:37.018005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 799/1608] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.018005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.078484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 810/1609] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:37.081815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4284291-f108-4804-bc98-2b0b7f864fe5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.083879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8231-8dc0-49d3-8b37-a9ea5ce00360 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.084176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8231-8dc0-49d3-8b37-a9ea5ce00360 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.084454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8231-8dc0-49d3-8b37-a9ea5ce00360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.087534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.088139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.088416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.088684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.089339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.091310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.115049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1bcc8231-8dc0-49d3-8b37-a9ea5ce00360 None None] MFA Rules not processed for user `2c01bfb1e8464616aceb3b73c958eb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:37.116353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1bcc8231-8dc0-49d3-8b37-a9ea5ce00360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:37.119367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 800/1610] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.120239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.183779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef12a-8703-405a-b64b-ede2a4178e10 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.186388 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 811/1611] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:37.187560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.187914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.188288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.188875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.189899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.190021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.190270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.190948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.192548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.206081 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 801/1612] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:37.206081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.206081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:37.219382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] MFA Rules not processed for user `3a1497a89b1a40acbb321ac88e483117`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:37.221264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:37.263929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaec90-d07d-4054-8b69-76cd5644c28f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.268356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.268952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.269227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.269497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.278463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:37.279315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.280747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.288646 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 812/1613] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:37.290448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.292491 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 802/1614] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:37.293321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.301303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955e02d-9575-41c9-ad48-ab32f96ecb69 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.302504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b027b71-3d8e-4d16-9f7d-b702d74c4f04 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.306446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.307706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.307816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.307924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.308086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.308177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.308389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.308663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.309340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.311249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.339290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 803/1615] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/users => generated 298 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.339290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.345609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:37.350815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7f5fc-b717-4f32-8940-0f7e7d748e9e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.359083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.360318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.360715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.361092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.361806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.364000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.379966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.381739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.393284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 813/1616] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:37.393964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.405219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abdd73-740d-4edb-9c23-1fa06c9e8035 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.406040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 804/1617] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.407225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.485440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.486069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.486348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.486659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.487295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.489317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.492822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9622-f007-4512-aaba-c0b9268a3d5c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.508280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.509397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.509749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.510425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.511103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.513840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.516024 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 814/1618] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.517482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.529294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220790b-19c8-494d-80d1-3de0922083d6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.535603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.536363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.536757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.536900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] PATH_INFO: `/v3/projects/79afe6f2d12f45e994b77b96c44f8239/users/64a47c4d366e4af88ff0386e79158e85/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.539821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79afe6f2d12f45e994b77b96c44f8239, user_id=64a47c4d366e4af88ff0386e79158e85, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.542801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.556347 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 805/1619] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.557329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.562262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 815/1620] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:37 2022] PUT /identity/v3/projects/79afe6f2d12f45e994b77b96c44f8239/users/64a47c4d366e4af88ff0386e79158e85/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:37.564202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.657954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaf713-d13f-4a5a-b189-121d856be5e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.664500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.665744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.666026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.666299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.667245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.669471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29581c-d1de-4546-974e-5e87e3bd9044 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.669596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.677932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.678762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.679222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.679599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.680392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.682327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.699274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 816/1621] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:37.699274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.708683 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 806/1622] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.708683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.710835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3f2f-67b1-4830-8175-856abdebbc86 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.715937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:37.786168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.786959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.787385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.787876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.788787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.790910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.796703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:37.815449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a072d-5e35-40d8-ad20-d3a2995b62b0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.821593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.822052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.822699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.822977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.823628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.825585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733d3d6-0c37-4167-9efb-169e38acbf21 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.832061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 817/1623] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.832061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.838738 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 807/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:37.840368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.840500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.840681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.845817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee29260-b405-472b-b026-2c969cf5104f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.853956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.854761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.855266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.855675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.856920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.862035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.874483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] MFA Rules not processed for user `64a47c4d366e4af88ff0386e79158e85`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:37.876971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:37.896952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 818/1625] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/users => generated 298 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:37.898582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.915206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc463e-1e00-4874-9c2c-136ec60b5872 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:37.921578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:37.922345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:37.922932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:37.923289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] PATH_INFO: `/v3/projects/91cba13ad0b14d8c9f622e112ede9e80/users/2bb371d8619b41f1b2bc508d543d3533/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:37.946756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cba13ad0b14d8c9f622e112ede9e80, user_id=2bb371d8619b41f1b2bc508d543d3533, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:37.950062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:37.966771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 808/1626] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:37.966771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.968166 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 819/1627] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:37 2022] PUT /identity/v3/projects/91cba13ad0b14d8c9f622e112ede9e80/users/2bb371d8619b41f1b2bc508d543d3533/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:37.968909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:37.993525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85960ca-36a7-47e8-b87c-1f8b1e60bc6a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.053791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f46cb-a35f-4f9d-815a-5eb02c06b9e4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.062075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.062728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.063182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.064004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.064829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.066328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.078681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 820/1628] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.079763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.099167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.099790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.099944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66a230-2f5c-44be-9922-3aef261423ef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.100054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.100328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.101328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.103658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.106272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.107456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.107901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.108372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] PATH_INFO: `/v3/projects/c316de3af02c4b9d996154c8db97ed1c/users/cd77dc79d7d14cc299ff36c71a34e36b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.130303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c316de3af02c4b9d996154c8db97ed1c, user_id=cd77dc79d7d14cc299ff36c71a34e36b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.133345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 809/1629] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:37 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:38.134055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.134828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.147139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe23d4-95ba-4630-973f-81c804402172 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.151951 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 821/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/c316de3af02c4b9d996154c8db97ed1c/users/cd77dc79d7d14cc299ff36c71a34e36b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.153871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.201412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:38.225716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.226525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.226813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.227185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] PATH_INFO: `/v3/users/af6db87401034d90bc8836a53414c517` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.228309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6db87401034d90bc8836a53414c517)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.230297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.236035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:38.244563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac23aa-5176-40a4-b38e-cb4d443f3fc6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.252382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.253456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.253947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.254288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.255948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.257789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.273244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 822/1631] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.274513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.290060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15844ee2-f667-4d3a-8077-1ec34b6d71b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.295649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.296343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.296929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.297353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.298714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.300565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc549b3-fda1-470f-a8ec-b5366187472d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.315276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 823/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.317877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6db87401034d90bc8836a53414c517'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:38.319926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.320223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.320501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.320817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6db87401034d90bc8836a53414c517'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:38.321780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 824/1633] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:38.322474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.324348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.336199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6db87401034d90bc8836a53414c517'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:38.350997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6db87401034d90bc8836a53414c517'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:38.353258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6db87401034d90bc8836a53414c517'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:38.356749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271bd6fc-38e7-4a1a-b69a-30bd57505719 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.362750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.363501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.364018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.364174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] PATH_INFO: `/v3/projects/cba9faf00ad847329f75483653ea5406/users/e5db797381db4f5c88444f8bbaf613c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.383507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba9faf00ad847329f75483653ea5406, user_id=e5db797381db4f5c88444f8bbaf613c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.386183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.389787 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 810/1634] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:38 2022] DELETE /identity/v3/users/af6db87401034d90bc8836a53414c517 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.390838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.393762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.407665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 825/1635] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/cba9faf00ad847329f75483653ea5406/users/e5db797381db4f5c88444f8bbaf613c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.408945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.412825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.479473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02303a-f7f0-4222-82be-1439697f367b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.503924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880e4c5-8dcb-4797-bb61-22e53891e84b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.509322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.510214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.510976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.511445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.512633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.514797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.534035 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 826/1636] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.536276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.538634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.554221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354980d-8aa8-40ce-a1be-29cf993c9ddf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.556123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.556788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.557185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.557530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.558242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.560725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.561240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.562095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.562596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.563063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] PATH_INFO: `/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7/users/c96cbcb3dc994e45a66930828065645a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.579786 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 811/1637] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.579786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.579786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.587291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1158a2f81d4d5ba36d8b1462cc5df7, user_id=c96cbcb3dc994e45a66930828065645a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.591780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b210e4-5649-49d5-8e28-1d85d912a3a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.596153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10821-5bd7-4488-b257-b91302db61dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.602542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.603180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.603467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.603737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] PATH_INFO: `/v3/projects/91cba13ad0b14d8c9f622e112ede9e80/users/2bb371d8619b41f1b2bc508d543d3533/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.606159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cba13ad0b14d8c9f622e112ede9e80, user_id=2bb371d8619b41f1b2bc508d543d3533, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.609753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56da32e-0d97-4b83-bf02-0484d5f84204 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.613385 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 827/1638] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7/users/c96cbcb3dc994e45a66930828065645a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.615457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.615812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.616201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.627424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 812/1639] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/91cba13ad0b14d8c9f622e112ede9e80/users/2bb371d8619b41f1b2bc508d543d3533/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.638379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.638986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.639470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.666435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:38.668613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.675631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:38.677093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.681368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:38.733412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:38.771072 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 813/1640] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:38.771770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.780249 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 828/1641] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:38.786447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ef2c6-d327-4161-9daa-4a6af764ca19 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.789286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.796351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.796964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.797234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.798188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.799273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.800966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.803560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557ca8-85ee-44be-be4b-8b5f125b85d3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.809678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.810365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.811369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.811369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] PATH_INFO: `/v3/projects/c316de3af02c4b9d996154c8db97ed1c/users/cd77dc79d7d14cc299ff36c71a34e36b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.813898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c316de3af02c4b9d996154c8db97ed1c, user_id=cd77dc79d7d14cc299ff36c71a34e36b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.817362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.823224 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 814/1642] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.824232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.836036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63524a-341f-442c-b4ef-06c907056acb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.839062 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 829/1643] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/c316de3af02c4b9d996154c8db97ed1c/users/cd77dc79d7d14cc299ff36c71a34e36b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.840377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.913767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.914719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.915093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.915485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] PATH_INFO: `/v3/projects/278fab1f5c894d09a9af383ac5bb6924/users/9e9c930f0d9a4996a4792165682ed2b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.926238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768fc4b-f25b-42ff-976b-3279d9d27a29 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:38.933013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:38.934451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.935036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.935595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.936605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.937887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278fab1f5c894d09a9af383ac5bb6924, user_id=9e9c930f0d9a4996a4792165682ed2b0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:38.938100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdef323-bb42-4984-9657-69fab89e5b1e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.941054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:38.959393 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 830/1644] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:38.959578 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 815/1645] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:38 2022] PUT /identity/v3/projects/278fab1f5c894d09a9af383ac5bb6924/users/9e9c930f0d9a4996a4792165682ed2b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:38.959578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:38.960630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:38.960929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:38.961054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:38.961204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:38.998147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] MFA Rules not processed for user `2bb371d8619b41f1b2bc508d543d3533`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:38.999461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:39.053165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f42e4-3910-4749-bca1-3aa1ff3b3a56 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:39.057257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:39.057859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.058129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.058401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.132367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 831/1646] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:39.132367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:39.133225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:39.187777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:39.190972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:39.211024 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 816/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:39.212504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:39.219281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:39.227094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a0f13-3dd4-4bb0-9580-096edf7e8ae5 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:39.232897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:39.233725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.234006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.234287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] PATH_INFO: `/v3/projects/cba9faf00ad847329f75483653ea5406/users/e5db797381db4f5c88444f8bbaf613c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.237151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba9faf00ad847329f75483653ea5406, user_id=e5db797381db4f5c88444f8bbaf613c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:39.241127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c2b15-2aba-48c7-b2da-65b33f4b5654 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:39.271271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11e8f1-d996-4766-9e46-83b237629824 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:39.289431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:39.291068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.291068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.291068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.299596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:39.314879 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 817/1648] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:39 2022] PUT /identity/v3/projects/cba9faf00ad847329f75483653ea5406/users/e5db797381db4f5c88444f8bbaf613c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:39.316372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.316659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.316934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.378610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] MFA Rules not processed for user `cd77dc79d7d14cc299ff36c71a34e36b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:39.381739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:39.471854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:39.471854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:39.494365 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 832/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:39 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:39.496072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:39.517217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feda13-5ea8-4f36-9321-ab917fce80c3 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:39.588939 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 818/1650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:39.588939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:39.708836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:39.709518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.709791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.710061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] PATH_INFO: `/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7/users/c96cbcb3dc994e45a66930828065645a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.717377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1158a2f81d4d5ba36d8b1462cc5df7, user_id=c96cbcb3dc994e45a66930828065645a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:39.725429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcda27b-b473-4db7-a9b6-99557693f5b3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:39.762463 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 833/1651] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:39 2022] PUT /identity/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7/users/c96cbcb3dc994e45a66930828065645a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:39.764325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b81ae-838b-48a4-9c3d-16689f54562e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.764814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b81ae-838b-48a4-9c3d-16689f54562e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.765288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b81ae-838b-48a4-9c3d-16689f54562e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.767544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 834/1652] 10.222.0.52 () {54 vars in 961 bytes} [Sat Sep 24 20:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:39.772448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3268ad8-23de-4625-a23e-102573518d18 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:39.774227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376a02-5b72-4e93-b70f-f30c39ad1828 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.774780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376a02-5b72-4e93-b70f-f30c39ad1828 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.775272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a376a02-5b72-4e93-b70f-f30c39ad1828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.841045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5a376a02-5b72-4e93-b70f-f30c39ad1828 None None] MFA Rules not processed for user `e5db797381db4f5c88444f8bbaf613c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:39.842912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a376a02-5b72-4e93-b70f-f30c39ad1828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:39.922595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:39.938271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:39.940227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.940768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.941296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:39.942807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:39.952176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc7a761-ed46-4a32-a685-b47516215bc4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:39.987515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:39.994205 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 819/1653] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:39.994205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:39.994205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:39.995331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.003406 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 835/1654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.004818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.005085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.005366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.049487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:40.050902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.069098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] MFA Rules not processed for user `c96cbcb3dc994e45a66930828065645a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:40.072263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.192165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 836/1655] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.193252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.194978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.210662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c004e61-127c-4cc5-be43-58508507f284 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.215933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.216794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.217538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.217951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] PATH_INFO: `/v3/projects/c80ef23e8f684c919bbe01fa65d145af` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.218684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 820/1656] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.220323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80ef23e8f684c919bbe01fa65d145af)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.220496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.222029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.231132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430f8b6-cb92-4aab-a8d2-bd08aafa2ed9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.236709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.237270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.237695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.237980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] PATH_INFO: `/v3/projects/278fab1f5c894d09a9af383ac5bb6924/users/9e9c930f0d9a4996a4792165682ed2b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.240126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278fab1f5c894d09a9af383ac5bb6924, user_id=9e9c930f0d9a4996a4792165682ed2b0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.243616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.260628 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 821/1657] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:40 2022] PUT /identity/v3/projects/278fab1f5c894d09a9af383ac5bb6924/users/9e9c930f0d9a4996a4792165682ed2b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:40.263356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.301465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80ef23e8f684c919bbe01fa65d145af', 'request_id': 'req-9277842f-9957-4099-b16b-2dc24c02ee9a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:40.310661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80ef23e8f684c919bbe01fa65d145af', 'request_id': 'req-9277842f-9957-4099-b16b-2dc24c02ee9a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:40.318381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80ef23e8f684c919bbe01fa65d145af', 'request_id': 'req-9277842f-9957-4099-b16b-2dc24c02ee9a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:40.337754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] The token cache is being invalidate because project c80ef23e8f684c919bbe01fa65d145af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:40.338067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:40.338818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:40.340914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 837/1658] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:40 2022] DELETE /identity/v3/projects/c80ef23e8f684c919bbe01fa65d145af => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:40.341826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.344194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.376086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f20ed-ade2-4674-9bda-b8b2f293cb93 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.449500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277842f-9957-4099-b16b-2dc24c02ee9a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.456361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.457130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.457480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.457816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.460247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.462034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:40.473499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:40.482277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.483293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.483766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.484221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.487909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.547069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.549071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b637c73-7864-41bd-8311-64b61994a680 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.560423 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 838/1659] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:40.561985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.562517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.563868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.573311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.574966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb47372-6753-4cac-8c2c-d3021509e573 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.600202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 822/1660] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:40.602235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827df09-25fc-4195-acb8-775904d7b47a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.603122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827df09-25fc-4195-acb8-775904d7b47a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.606822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827df09-25fc-4195-acb8-775904d7b47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.607313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] MFA Rules not processed for user `9e9c930f0d9a4996a4792165682ed2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:40.608736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.679716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4827df09-25fc-4195-acb8-775904d7b47a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:40.680962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4827df09-25fc-4195-acb8-775904d7b47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.801115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 839/1661] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.801115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.802971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.816195 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 823/1662] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.825219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.825905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.825905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.834456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bfa17-895f-4a9a-b8f9-3243d729afef None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.841026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.842765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.843048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.843323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.846331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.869564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:40.871074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:40.893917 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 824/1663] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:40.897669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.913485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3675-f7b3-4e8f-a821-f05bb1038511 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.919959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.920821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.921279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.921728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.922640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:40.923140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.925092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.940045 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 825/1664] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:40.945327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:40.961824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.965624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.966977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154a931-3ffc-47ff-8762-371ed9369714 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:40.977452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:40.978072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:40.978346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:40.978649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:40.983222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:40.985596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:40.987578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 840/1665] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:41.053726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.055503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 826/1666] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:40 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:49:41.059549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.074226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcb31c-c744-4909-aff6-2fe723d47192 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.132796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:41.180290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:41.201875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f46e39-fb00-41b3-8782-e447fdc65c30 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.216945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.217567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.217848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.218118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.218798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.221326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.249050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.249740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.252365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.252752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.254746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:41.274305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 827/1667] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:41 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:41.275207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.302588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709dc6-7165-42f8-8798-17431a441dbd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.315230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.315854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.316132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.316405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.357948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.360518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.375091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 841/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:49:41.376667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.389114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146aba-7514-492c-9640-5a882b5d746b service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.400704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.401337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.401616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.401887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.404549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.405829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.408131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.408712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.434972 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 842/1669] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:41.437641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.437878 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 828/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:41.461260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9f400-4159-4622-b266-e6722dc812ce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.475954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.488082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.488380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.488663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.489351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.498945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103ad5e-a5b3-4c1f-b0de-50092ddddc87 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.524453 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 843/1671] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:41.525870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.541698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526025-96ab-4ffb-9a4b-b06186447031 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.555772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.556455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.556732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.557002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] PATH_INFO: `/v3/projects/fbe5c2862f3b4ae58fcfc0103efb4a1a/users/598d337daeb4481991e2591ad710d79c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.599527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe5c2862f3b4ae58fcfc0103efb4a1a, user_id=598d337daeb4481991e2591ad710d79c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.606044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce96d69-19a8-4645-beb7-84e499bacc73 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.640266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09c52-ec9b-423b-bb93-dfc252523207 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.641372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09c52-ec9b-423b-bb93-dfc252523207 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.641648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a09c52-ec9b-423b-bb93-dfc252523207 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.643721 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 844/1672] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:41.649712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.649932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.650207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.664164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 829/1673] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:41 2022] PUT /identity/v3/projects/fbe5c2862f3b4ae58fcfc0103efb4a1a/users/598d337daeb4481991e2591ad710d79c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:49:41.678836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.681400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.681683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.699723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:41.701062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:41.731597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] MFA Rules not processed for user `598d337daeb4481991e2591ad710d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:41.733039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:41.810635 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 845/1674] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:41.845988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.847936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:41.869299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 830/1675] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:41.870447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.881429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369bfdf-e947-48eb-8cba-4184b4dbb95c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.898601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.899238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.899519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.899788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.917339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.921467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.941042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:41.967968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e88c6-b272-423d-8783-944cea288f91 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:41.970006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 846/1676] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:49:41.973585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:41.974751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:41.975675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:41.975951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:41.976237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:41.977369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:41.979865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:41.980150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.015242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b053e-448b-4a8d-a101-edf5565afdf5 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.029247 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 831/1677] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:41 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:42.038399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.039079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.040269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.040269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] PATH_INFO: `/v3/users/3a1497a89b1a40acbb321ac88e483117` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.043292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.052832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1497a89b1a40acbb321ac88e483117)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:42.057035 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:42.057035 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 20:49:42.057035 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 20:49:42.064532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:42.098776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:42.194189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf359c-83b1-4728-a84b-d0e2f1fa2c3f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.212442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.213066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.213338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.213597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.214229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:42.216522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:42.240801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1497a89b1a40acbb321ac88e483117'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.244189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1497a89b1a40acbb321ac88e483117'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.251424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 832/1678] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:42 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:42.252353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.256584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.269216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1497a89b1a40acbb321ac88e483117'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.292097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe349f-aa54-4df0-922b-fb58d09abc59 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.302717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1497a89b1a40acbb321ac88e483117'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.305320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1497a89b1a40acbb321ac88e483117'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.305482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.306115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.306395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.306741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] PATH_INFO: `/v3/users/08746bd50eda47eaa632fb972b4d2097` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.314713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08746bd50eda47eaa632fb972b4d2097)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:42.317865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:42.343304 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 847/1679] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:41 2022] DELETE /identity/v3/users/3a1497a89b1a40acbb321ac88e483117 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:49:42.344163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.347440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.406927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08746bd50eda47eaa632fb972b4d2097'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.408429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08746bd50eda47eaa632fb972b4d2097'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.432776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08746bd50eda47eaa632fb972b4d2097'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.444582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08746bd50eda47eaa632fb972b4d2097'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.445613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-365bab10-823f-4ebe-ba3c-8e9d98a8a495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08746bd50eda47eaa632fb972b4d2097'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:42.485932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8c2d-526c-439e-ab0d-8251cd043148 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.507365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 833/1680] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:42 2022] DELETE /identity/v3/users/08746bd50eda47eaa632fb972b4d2097 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:42.507365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.507365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.507365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.507901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.507901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.523883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.524418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.526248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:42.528414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74643dd0-a6bf-4a13-9b9f-030364efdfe3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:42.548943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:42.550294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.567033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 848/1681] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:42.575103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416358-5300-4e35-b4c9-d37bdccfb041 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.577217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416358-5300-4e35-b4c9-d37bdccfb041 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.578190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416358-5300-4e35-b4c9-d37bdccfb041 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.584802 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 849/1682] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:49:42 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:42.586991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.587492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.588816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.589206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:42.619776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:42.621476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.643696 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 834/1683] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:42.643696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.643696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.657072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdf6fe-bf71-47fd-b53d-9ee8dd8bb2c8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.663137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.663927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.664209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.664528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] PATH_INFO: `/v3/projects/b34e5423681c493eabeb7189cb3acb91/users/c3d50d2fa3d642ec93ea44d8d0a2c96e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.693203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e5423681c493eabeb7189cb3acb91, user_id=c3d50d2fa3d642ec93ea44d8d0a2c96e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:42.694422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab265ee1-528e-41e8-baf4-cbe62fa3cdac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:42.718191 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 835/1684] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:42 2022] PUT /identity/v3/projects/b34e5423681c493eabeb7189cb3acb91/users/c3d50d2fa3d642ec93ea44d8d0a2c96e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:42.720029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.720420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.720798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:42.747272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:42.748665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.774937 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 850/1685] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:42.775845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.780039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:42.836208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:42.884202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195811-5487-4c09-9c7b-60b811df07c8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:42.896056 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 836/1686] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:49:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:42.898781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:42.966817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:42.967498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:42.968344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:42.968681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.028399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7a01a-3dc2-4ef7-bf80-0ba83161a96d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.047075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.048071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.048370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.048809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.049767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.051494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.071148 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 837/1687] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:43.073043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.093399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.099997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.120037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847baa0e-1f1f-4c59-a0c0-999a6ebe7de3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.125027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 851/1688] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:43.126325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.127571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.128452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.128945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.129323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.130007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:43.141838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.143230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.147264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195eee8-ca24-48f5-b7a1-3f7f8c2ab3ef service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.154673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.157628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.158117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.161000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 838/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:43.162706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.165040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.166749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:43.180748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.185573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.200187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 852/1690] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:43.200852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.228945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2b44-dc45-4522-9ce1-c4ac32d25410 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.247551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.248117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.248395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.248667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] PATH_INFO: `/v3/projects/b34e5423681c493eabeb7189cb3acb91/users/c3d50d2fa3d642ec93ea44d8d0a2c96e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.251593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34e5423681c493eabeb7189cb3acb91, user_id=c3d50d2fa3d642ec93ea44d8d0a2c96e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.255462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6233d22-b739-4cd9-8424-1abfd0d1d7a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.272427 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 853/1691] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:43 2022] PUT /identity/v3/projects/b34e5423681c493eabeb7189cb3acb91/users/c3d50d2fa3d642ec93ea44d8d0a2c96e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:43.283457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.283746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.284023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.285118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:43.340071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] MFA Rules not processed for user `c3d50d2fa3d642ec93ea44d8d0a2c96e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:43.343832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:43.424426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ab532-c367-4c98-9dc6-e9063ccae6e8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.513776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:43.575517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 854/1692] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:43.580640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.581263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.581539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.581833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.583982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:43.659858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.661683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:43.715437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352a4b4-dd01-4960-bd56-12bc9f451a6d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.738991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.739888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.741286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.741873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.782369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.785445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:43.792933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.794054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:43.819164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 839/1693] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:43.819690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 855/1694] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:43.868238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:43.898553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8b3e-66ce-4867-be08-1967ba03dfa7 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:43.910384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:43.911089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:43.911324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:43.911593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:43.917133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:44.027789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:44.029942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:44.125203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:44.125203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:44.132246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:44.139620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 840/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:44.140440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:44.159011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0fffc-4204-46fe-b17d-9c3e4c63d820 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:44.172684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:44.173455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:44.173736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:44.173999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:44.176441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:44.207121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae399f53-e697-4313-afa2-a10ae2aad2a0 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:44.213205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:44.214216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:44.214521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:44.214812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] PATH_INFO: `/v3/projects/d29eee0ecb094c65ab13e3d567d3ddf5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:44.216005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29eee0ecb094c65ab13e3d567d3ddf5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:44.218613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:44.298809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:44.300535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:44.316934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 841/1696] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:44.321294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29eee0ecb094c65ab13e3d567d3ddf5', 'request_id': 'req-51cff2d1-99ec-497d-9a88-f99e872e18c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.331523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29eee0ecb094c65ab13e3d567d3ddf5', 'request_id': 'req-51cff2d1-99ec-497d-9a88-f99e872e18c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.346540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29eee0ecb094c65ab13e3d567d3ddf5', 'request_id': 'req-51cff2d1-99ec-497d-9a88-f99e872e18c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.371657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] The token cache is being invalidate because project d29eee0ecb094c65ab13e3d567d3ddf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:44.372179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.373047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.376509 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 856/1697] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:44 2022] DELETE /identity/v3/projects/d29eee0ecb094c65ab13e3d567d3ddf5 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:49:44.381231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:44.393754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:44.398202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:44.403791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af7d4f-bb6a-4cc7-8418-46c383e9054c service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:44.545316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:44.546061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:44.546337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:44.546632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] PATH_INFO: `/v3/users/9e9c930f0d9a4996a4792165682ed2b0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:44.550160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9c930f0d9a4996a4792165682ed2b0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:44.553551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:44.584438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cff2d1-99ec-497d-9a88-f99e872e18c6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:44.589827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:44.590476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:44.590926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:44.591198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] PATH_INFO: `/v3/projects/85ca75b9c5844ba28780c8218a113a86` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:44.592902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ca75b9c5844ba28780c8218a113a86)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:44.595478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:44.610771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:44.690054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9c930f0d9a4996a4792165682ed2b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.690054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9c930f0d9a4996a4792165682ed2b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.707172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9c930f0d9a4996a4792165682ed2b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.721261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9c930f0d9a4996a4792165682ed2b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.723201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9c930f0d9a4996a4792165682ed2b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.730553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ca75b9c5844ba28780c8218a113a86', 'request_id': 'req-db365666-45c4-4788-9488-11260851c0f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.747153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ca75b9c5844ba28780c8218a113a86', 'request_id': 'req-db365666-45c4-4788-9488-11260851c0f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.762991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ca75b9c5844ba28780c8218a113a86', 'request_id': 'req-db365666-45c4-4788-9488-11260851c0f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.792412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] The token cache is being invalidate because project 85ca75b9c5844ba28780c8218a113a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:44.792712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.794250 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 842/1698] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:44 2022] DELETE /identity/v3/users/9e9c930f0d9a4996a4792165682ed2b0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:44.794828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:44.799735 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 857/1699] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:44 2022] DELETE /identity/v3/projects/85ca75b9c5844ba28780c8218a113a86 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:44.804495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:44.805991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:44.858898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:44.990087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4db03-d164-444a-9285-4da0225c0b61 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:45.009627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:45.015579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.015923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.016222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] PATH_INFO: `/v3/users/e5db797381db4f5c88444f8bbaf613c1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.020631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5db797381db4f5c88444f8bbaf613c1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:45.026011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:45.114138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:45.115792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:45.159329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5db797381db4f5c88444f8bbaf613c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.161110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5db797381db4f5c88444f8bbaf613c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.176537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5db797381db4f5c88444f8bbaf613c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.195106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5db797381db4f5c88444f8bbaf613c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.197791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5db797381db4f5c88444f8bbaf613c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.251779 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 843/1700] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:44 2022] DELETE /identity/v3/users/e5db797381db4f5c88444f8bbaf613c1 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:45.253498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:45.255250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:45.257136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365666-45c4-4788-9488-11260851c0f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:45.341275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:45.341945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.342663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.342777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] PATH_INFO: `/v3/users/598d337daeb4481991e2591ad710d79c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.344101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=598d337daeb4481991e2591ad710d79c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:45.346852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:45.368343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc641a-a17b-4b52-8c7b-8faebd64ae58 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:45.374436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:45.375211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.376977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.377331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] PATH_INFO: `/v3/users/09dcd838c264480aab44daca185b5769` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.379074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09dcd838c264480aab44daca185b5769)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:45.381759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:45.472483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598d337daeb4481991e2591ad710d79c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.473434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598d337daeb4481991e2591ad710d79c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.476979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:45.504924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598d337daeb4481991e2591ad710d79c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.522719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:45.543797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598d337daeb4481991e2591ad710d79c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.546429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a1d7abc-494e-4dfb-aac1-f174e12d32d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598d337daeb4481991e2591ad710d79c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.552000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09dcd838c264480aab44daca185b5769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.553416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09dcd838c264480aab44daca185b5769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.571020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09dcd838c264480aab44daca185b5769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.587283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09dcd838c264480aab44daca185b5769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.588037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09dcd838c264480aab44daca185b5769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:45.590426 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 858/1701] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:45 2022] DELETE /identity/v3/users/598d337daeb4481991e2591ad710d79c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:45.591778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa0875-8379-467b-a466-647aa9857848 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.592057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa0875-8379-467b-a466-647aa9857848 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.592337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa0875-8379-467b-a466-647aa9857848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.634375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3aa0875-8379-467b-a466-647aa9857848 None None] MFA Rules not processed for user `d278977b201c48dda2d73823fbfe4833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:45.636506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3aa0875-8379-467b-a466-647aa9857848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:45.641175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 844/1702] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:45 2022] DELETE /identity/v3/users/09dcd838c264480aab44daca185b5769 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:45.642188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:45.645304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:45.784266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9257-09de-4490-91df-c5f123306941 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:45.797299 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 859/1703] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:45.799073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdfd57-0393-49a7-90c3-92af532ec92c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.799442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdfd57-0393-49a7-90c3-92af532ec92c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.799617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdfd57-0393-49a7-90c3-92af532ec92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.803906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:45.806852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.807222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.809849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] PATH_INFO: `/v3/projects/278fab1f5c894d09a9af383ac5bb6924` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.814482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=278fab1f5c894d09a9af383ac5bb6924)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:45.817932 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 20:49:45.817932 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 20:49:45.817932 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 20:49:45.822487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:45.857348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-facdfd57-0393-49a7-90c3-92af532ec92c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:45.862972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-facdfd57-0393-49a7-90c3-92af532ec92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:45.886990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 860/1704] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:45.888251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af4b8-1097-4c05-8a7d-30652a38f8ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:45.888538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af4b8-1097-4c05-8a7d-30652a38f8ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:45.888808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855af4b8-1097-4c05-8a7d-30652a38f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:45.943823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-855af4b8-1097-4c05-8a7d-30652a38f8ca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:45.947658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-855af4b8-1097-4c05-8a7d-30652a38f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.051717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278fab1f5c894d09a9af383ac5bb6924', 'request_id': 'req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.071746 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 861/1705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:46.073201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.073491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.073764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.076735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278fab1f5c894d09a9af383ac5bb6924', 'request_id': 'req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.104661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278fab1f5c894d09a9af383ac5bb6924', 'request_id': 'req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.114224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] MFA Rules not processed for user `2c01bfb1e8464616aceb3b73c958eb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:46.116292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.146767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] The token cache is being invalidate because project 278fab1f5c894d09a9af383ac5bb6924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:46.146767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.148347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.157244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 845/1706] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:45 2022] DELETE /identity/v3/projects/278fab1f5c894d09a9af383ac5bb6924 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:46.157997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.160141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.186801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f97f0-0d99-425e-8401-fb3d6d6ffd2b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.194833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.195659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.196079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.196402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.197151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:46.199333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:46.226665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:46.243856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 846/1707] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:46 2022] POST /identity/v3/projects => generated 401 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:46.244836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.246402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.258156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 862/1708] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:46.259310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.260964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.296633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:46.349048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba712-78b2-4177-9844-466455ad157f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.349391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a1305-6994-49e3-91da-8bf2069c1002 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.353379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.354190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.354299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.354515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.354818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.355205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.355482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.355751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.356825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.358792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.443799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:46.445509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65991e3-e207-4d54-a549-85260d1b40ae service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:46.455806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:46.457507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:46.458644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 847/1709] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:46.471140 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 863/1710] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:46.473261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.473575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.473836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.473938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.476347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.505314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae40b-6558-4ed9-8a29-bb8bec139792 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.520839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:46.521570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.522221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.522519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.522905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] PATH_INFO: `/v3/projects/cba9faf00ad847329f75483653ea5406` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.524485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.525689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba9faf00ad847329f75483653ea5406)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:46.528010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:46.551791 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 848/1711] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:46.563956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.597416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab3e00-729c-449f-9206-098cd1fad0b5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.604804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.605871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.606283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.606884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.607567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:46.613228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:46.671785 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 849/1712] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:46 2022] POST /identity/v3/users => generated 307 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:46.673106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.675384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba9faf00ad847329f75483653ea5406', 'request_id': 'req-43296ea6-83d5-4059-94f5-25ad517a54f9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.680478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.690991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba9faf00ad847329f75483653ea5406', 'request_id': 'req-43296ea6-83d5-4059-94f5-25ad517a54f9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.713393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba9faf00ad847329f75483653ea5406', 'request_id': 'req-43296ea6-83d5-4059-94f5-25ad517a54f9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.741590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] The token cache is being invalidate because project cba9faf00ad847329f75483653ea5406 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:46.741904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.746326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43296ea6-83d5-4059-94f5-25ad517a54f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:46.752496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 864/1713] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:46 2022] DELETE /identity/v3/projects/cba9faf00ad847329f75483653ea5406 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:46.754489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.754801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.755092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.816344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:46.819266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.826041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e45f37-5dcc-41b4-8478-b02fcf60b1c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:46.866527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:46.887549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:46.912614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:46.913402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:46.913744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:46.914084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:46.915999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.960549 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 865/1714] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:46.961447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:46.966424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:46.992531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abca77-1751-47f9-99ad-2de427ad13f1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.005324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.005953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.006224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.006490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.007152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.008767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.011119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.012718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026416-7c2f-4eaa-b1a5-0b68f38f58f3 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.025754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 850/1715] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:47.027378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.027954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.028431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.032957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 866/1716] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:47.032957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.033522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.049006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f51f97-c82d-4837-b02f-143f6f15cb89 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.054740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.055378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.055653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.055928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] PATH_INFO: `/v3/users/fb4bd7ce915e4b329d3f8cf1ee5f2d3f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.057059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4bd7ce915e4b329d3f8cf1ee5f2d3f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.059141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.066360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:47.067893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.078904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 851/1717] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:47.080321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.091917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192da19-2551-496b-9d70-534b1e7bb77c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.098084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.098940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.099386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.099817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.100621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:47.101128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.102932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.118569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 852/1718] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:47.119666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.121396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.140376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de37ab-4cf7-4186-8d77-93704274498e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.147654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.148306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.148593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.148995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] PATH_INFO: `/v3/projects/dcbc4f38ed72401396f32f8ab0261472` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.150466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbc4f38ed72401396f32f8ab0261472)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.152421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.160385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4bd7ce915e4b329d3f8cf1ee5f2d3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.161803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4bd7ce915e4b329d3f8cf1ee5f2d3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.174797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4bd7ce915e4b329d3f8cf1ee5f2d3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.194186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4bd7ce915e4b329d3f8cf1ee5f2d3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.194992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4bd7ce915e4b329d3f8cf1ee5f2d3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.213583 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 867/1719] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:47 2022] DELETE /identity/v3/users/fb4bd7ce915e4b329d3f8cf1ee5f2d3f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:47.219951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.226797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.231378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbc4f38ed72401396f32f8ab0261472', 'request_id': 'req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.240729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbc4f38ed72401396f32f8ab0261472', 'request_id': 'req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.252337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbc4f38ed72401396f32f8ab0261472', 'request_id': 'req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.265442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] The token cache is being invalidate because project dcbc4f38ed72401396f32f8ab0261472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:47.265827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.266759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d3f35643-cdc4-47fe-93a1-4153e2f7e57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.268763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 853/1720] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:47 2022] DELETE /identity/v3/projects/dcbc4f38ed72401396f32f8ab0261472 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:47.270408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.270988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.271346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.295881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:47.297948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.306362 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 854/1721] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:47.309172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.310700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.316135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c22194-1e4a-49a4-8f35-6bb172bdeb9d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.321905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.322603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.322919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.323196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] PATH_INFO: `/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c/users/0686ece5b3ea46269d1098999ba98330/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.323614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded3802-20a0-4e53-9569-602895aa6815 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.329724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.330336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.330654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.331076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.331968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.334270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4bb1-68b4-4310-9fb6-9e5a92f1b5e7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.340517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197a3a5ec6ca4368a1fd953d2dbfd47c, user_id=0686ece5b3ea46269d1098999ba98330, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.345496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86208d3-d436-4385-8e5d-c5946b76f945 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.362789 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 868/1722] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:47 2022] PUT /identity/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c/users/0686ece5b3ea46269d1098999ba98330/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:47.364344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.364687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.364878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.366220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 869/1723] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:47.367733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.369253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.376106 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 855/1724] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:47.377728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.378021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.378295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.389078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:47.398990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:47.400477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.416475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:47.463122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547dc3d-faba-4a89-b338-afdd21b9feac None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.479078 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 856/1725] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:47.479906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.481185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.495625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8795e5-4ba5-49e3-a7f1-dee45ab315bc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.500892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.501532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.502794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.502794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] PATH_INFO: `/v3/users/2bb371d8619b41f1b2bc508d543d3533` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.503850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb371d8619b41f1b2bc508d543d3533)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.505913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.533534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.534164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.534440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.534752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] PATH_INFO: `/v3/projects/fbe5c2862f3b4ae58fcfc0103efb4a1a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.535873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe5c2862f3b4ae58fcfc0103efb4a1a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.538130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.563571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb371d8619b41f1b2bc508d543d3533'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.564177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb371d8619b41f1b2bc508d543d3533'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.574189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb371d8619b41f1b2bc508d543d3533'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.582929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb371d8619b41f1b2bc508d543d3533'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.583605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb371d8619b41f1b2bc508d543d3533'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.599154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe5c2862f3b4ae58fcfc0103efb4a1a', 'request_id': 'req-abfadd5c-b08b-416d-a69c-f122e06124c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.602314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 857/1726] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:47 2022] DELETE /identity/v3/users/2bb371d8619b41f1b2bc508d543d3533 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:47.606516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.607871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.610921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe5c2862f3b4ae58fcfc0103efb4a1a', 'request_id': 'req-abfadd5c-b08b-416d-a69c-f122e06124c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.623866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe5c2862f3b4ae58fcfc0103efb4a1a', 'request_id': 'req-abfadd5c-b08b-416d-a69c-f122e06124c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.652951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] The token cache is being invalidate because project fbe5c2862f3b4ae58fcfc0103efb4a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:47.653348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.654635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:47.661627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 870/1727] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:47 2022] DELETE /identity/v3/projects/fbe5c2862f3b4ae58fcfc0103efb4a1a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:47.661627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.665554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.702682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9bf46e-3e39-4498-a092-6c4aa0363776 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.709050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.709810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.710151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.710439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.711130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:47.711435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.714693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d8805-88d0-4e8b-925e-b4e38f23e0a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.735650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 858/1728] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:47.735650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.735650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.735650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.751194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:47.758921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:47.760201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.767329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfadd5c-b08b-416d-a69c-f122e06124c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.772937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.773630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.773989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.774839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.775878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.778139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.784464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:47.792229 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 871/1729] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:47.792229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.796660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.818837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38a42b-4c40-4ad0-a391-3a6869709e2b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.826132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.826976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.827576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.827694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.828553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.830888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.864060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 872/1730] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/users => generated 294 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:47.865639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.868709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.895675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 859/1731] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:47.898079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.899804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.917923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029725-67c6-4e27-a553-3f6eb9d3170b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.926676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.927589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.928117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.928636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] PATH_INFO: `/v3/users/c3d50d2fa3d642ec93ea44d8d0a2c96e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.930679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d50d2fa3d642ec93ea44d8d0a2c96e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.932766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.945420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a5498-0b0b-4392-8009-b781ab6ae05b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:47.950454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:47.952033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:47.952424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:47.952799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:47.955681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:47.974695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:47.976952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:47.990306 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 873/1732] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:47.991319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:47.993003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.008597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b650e6-b309-4002-a5bb-67fd9284e871 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.014613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.015332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.017841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.018086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.019279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.022212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.023190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d50d2fa3d642ec93ea44d8d0a2c96e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.024568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d50d2fa3d642ec93ea44d8d0a2c96e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.043114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d50d2fa3d642ec93ea44d8d0a2c96e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.057661 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 874/1733] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:47 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:48.058645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.060947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.065244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d50d2fa3d642ec93ea44d8d0a2c96e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.066258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d50d2fa3d642ec93ea44d8d0a2c96e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.099583 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 860/1734] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:47 2022] DELETE /identity/v3/users/c3d50d2fa3d642ec93ea44d8d0a2c96e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:48.101181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.103205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.204654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57739-2fbd-4e00-8978-9bcf704d4c91 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.262862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b351eaf-822c-468b-9570-aa7052aaa8ae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.286641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.288356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.288819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.291506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.293571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.300181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.338022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 861/1735] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:48.341268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.342292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.356214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:48.368676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.371100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.371552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.372118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] PATH_INFO: `/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c/users/0686ece5b3ea46269d1098999ba98330/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.376464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=197a3a5ec6ca4368a1fd953d2dbfd47c, user_id=0686ece5b3ea46269d1098999ba98330, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.380095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.405420 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 875/1736] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:48 2022] PUT /identity/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c/users/0686ece5b3ea46269d1098999ba98330/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:48.409119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.410615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:48.412319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.511761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563335-53b2-4e3f-babf-b4181864c15c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.518674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.519013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.519632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.519632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] PATH_INFO: `/v3/projects/91cba13ad0b14d8c9f622e112ede9e80` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.520718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cba13ad0b14d8c9f622e112ede9e80)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.523480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.543427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebad132-654a-4e35-999b-965b7a0c861e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.588058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cba13ad0b14d8c9f622e112ede9e80', 'request_id': 'req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.595309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cba13ad0b14d8c9f622e112ede9e80', 'request_id': 'req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.604581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cba13ad0b14d8c9f622e112ede9e80', 'request_id': 'req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.627141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] The token cache is being invalidate because project 91cba13ad0b14d8c9f622e112ede9e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:48.627812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.628020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.628948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.629244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.629520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.629709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.631517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.632263 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 876/1737] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:48 2022] DELETE /identity/v3/projects/91cba13ad0b14d8c9f622e112ede9e80 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:48.633335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.635086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.743435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.745789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.753824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f78a3-c1f3-4c52-9dc9-6fb6ccf960f6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.758877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.759657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.760125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.761733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.762559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.764195 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 862/1738] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:48.764996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.782587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.786853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.796259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 877/1739] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:48 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:48.796259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.799560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.816850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41cc4-3396-4bfd-a0a8-10960dd82bc5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.817416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60e688-66fc-471a-8f41-740ed3be1e86 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.825697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.826019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.826670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.827000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.827305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.827720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.828143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.829224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.829697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] PATH_INFO: `/v3/projects/084b640ae5ed4ce683962557ecdd84ec` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.833128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084b640ae5ed4ce683962557ecdd84ec)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.835578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.838856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:48.841518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.860770 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 863/1740] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:48.861848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:48.864058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:48.885455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bff54-612d-4f67-b33f-3547692e7293 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:48.892621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:48.893277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.893644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.894015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] PATH_INFO: `/v3/projects/b34e5423681c493eabeb7189cb3acb91` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:48.895924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34e5423681c493eabeb7189cb3acb91)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:48.898356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:48.936400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084b640ae5ed4ce683962557ecdd84ec', 'request_id': 'req-dd99895f-24c6-4510-8559-3db3fa3f2725'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.947916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084b640ae5ed4ce683962557ecdd84ec', 'request_id': 'req-dd99895f-24c6-4510-8559-3db3fa3f2725'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.962773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084b640ae5ed4ce683962557ecdd84ec', 'request_id': 'req-dd99895f-24c6-4510-8559-3db3fa3f2725'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.970323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:48.976733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34e5423681c493eabeb7189cb3acb91', 'request_id': 'req-e21517b1-9cb1-4a04-a181-c2577ac5e497'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.985037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] The token cache is being invalidate because project 084b640ae5ed4ce683962557ecdd84ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:48.985494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.986880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd99895f-24c6-4510-8559-3db3fa3f2725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.989211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34e5423681c493eabeb7189cb3acb91', 'request_id': 'req-e21517b1-9cb1-4a04-a181-c2577ac5e497'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:48.991009 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 878/1741] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:48 2022] DELETE /identity/v3/projects/084b640ae5ed4ce683962557ecdd84ec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:48.996096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:48.996455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:48.997118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.000131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34e5423681c493eabeb7189cb3acb91', 'request_id': 'req-e21517b1-9cb1-4a04-a181-c2577ac5e497'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.019082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] The token cache is being invalidate because project b34e5423681c493eabeb7189cb3acb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:49.019376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.020063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.025858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 864/1742] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:48 2022] DELETE /identity/v3/projects/b34e5423681c493eabeb7189cb3acb91 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:49.025858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.025858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.034399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:49.036089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.105781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21517b1-9cb1-4a04-a181-c2577ac5e497 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.113311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.114053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.114465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.114862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.115607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.117442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.118093 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 879/1743] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:49.127905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.127905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.134981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090be1-eda7-4b02-9006-c007bc64698c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.139602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.140266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.140571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.141736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] PATH_INFO: `/v3/projects/c3800968cd944059a397e9c6b35f976d/users/50d8a91794144d73a8942f5c3bac8cc8/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.143104 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 865/1744] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:49.144510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.148497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.160716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3800968cd944059a397e9c6b35f976d, user_id=50d8a91794144d73a8942f5c3bac8cc8, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.164672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc8a89-7d20-49ae-833f-092d9adb6c74 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.171339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21e66-0f4a-4b62-b9aa-e76e399cc6ad admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.178980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.180097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.180518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.180971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] PATH_INFO: `/v3/users/cd77dc79d7d14cc299ff36c71a34e36b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.181263 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 880/1745] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:49 2022] PUT /identity/v3/projects/c3800968cd944059a397e9c6b35f976d/users/50d8a91794144d73a8942f5c3bac8cc8/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:49.182885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd77dc79d7d14cc299ff36c71a34e36b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.183018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.183380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.183718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.185415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.222391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:49.224001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.263393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd77dc79d7d14cc299ff36c71a34e36b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.266205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd77dc79d7d14cc299ff36c71a34e36b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.283461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd77dc79d7d14cc299ff36c71a34e36b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.298591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd77dc79d7d14cc299ff36c71a34e36b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.300640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b56ab7f-41f0-4299-8ae7-bacbbdba1d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd77dc79d7d14cc299ff36c71a34e36b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:49.344519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 866/1746] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:49 2022] DELETE /identity/v3/users/cd77dc79d7d14cc299ff36c71a34e36b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:49.346427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a3e1-b57f-4480-a4ac-34b38e16076f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.346813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a3e1-b57f-4480-a4ac-34b38e16076f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.347194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70a3e1-b57f-4480-a4ac-34b38e16076f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.370296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 881/1747] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:49.370296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.370745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.371465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2c70a3e1-b57f-4480-a4ac-34b38e16076f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:49.372731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2c70a3e1-b57f-4480-a4ac-34b38e16076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.386127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee822fd-3c2a-41e0-a76d-3b154e3521b1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.392180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:49.427963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:49.470362 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 867/1748] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:49.474729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.476170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.476730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.479622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.480322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.480813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.480949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.482102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.484044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4ad7-bfde-4ea7-8d11-62ecdf2917c5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.498359 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 882/1749] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:49.500797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.501066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.501358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.534163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:49.535771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.544568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] MFA Rules not processed for user `50d8a91794144d73a8942f5c3bac8cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:49.546184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.568686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 883/1750] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:49.574041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.601031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779a6a-457c-4ee4-84ee-ae6ad3daa596 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.615863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.616489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.616882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.617149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.617914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:49.618211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.620100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.657368 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 868/1751] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:49.657838 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 884/1752] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:49.658607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.658769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.675663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5317c40-c83d-4dc3-b3f3-c35a75554f62 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.683389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aaeb4-dbcb-4a08-8016-5056969461a4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.691494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.692471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.692895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.693315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] PATH_INFO: `/v3/projects/03b8651c3c7d4ae6a200f8613a96c840/users/9f3c11be6cc84332b079da481c6820b7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.696624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.696911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.697188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.701604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.702484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:49.707012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.716013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.727046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b8651c3c7d4ae6a200f8613a96c840, user_id=9f3c11be6cc84332b079da481c6820b7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.740965 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 885/1753] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:49.741186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.741277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.743612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:49.775550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 869/1754] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:49 2022] PUT /identity/v3/projects/03b8651c3c7d4ae6a200f8613a96c840/users/9f3c11be6cc84332b079da481c6820b7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:49.776777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:49.821003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715bd5d-59fa-422d-a7ed-acfbee55271a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.940202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.942981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.943297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.943947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.946127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.950814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:49.958296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2b15c-cde0-4c05-85fa-5d70d3cd1ad7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:49.969859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:49.970532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:49.970809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:49.971085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:49.971727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:49.977244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882ee66-c85c-4e09-8279-fe114b9e3a5a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.020348 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 886/1755] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/projects => generated 389 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.021528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.085586 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 870/1756] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:49 2022] POST /identity/v3/projects => generated 381 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.093710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64ceb-fe8c-4e1b-91fa-af030998505a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.097102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64ceb-fe8c-4e1b-91fa-af030998505a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.098381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64ceb-fe8c-4e1b-91fa-af030998505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.124758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:50.129524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7445c-5b4f-4ba9-8682-b9bdd5d0bf77 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.164641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7ba64ceb-fe8c-4e1b-91fa-af030998505a None None] MFA Rules not processed for user `9f3c11be6cc84332b079da481c6820b7`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:50.166571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7ba64ceb-fe8c-4e1b-91fa-af030998505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.215247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:50.245391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.248893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.249477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.250463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.251313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.254178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.318883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 871/1757] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:50.320577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.321025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.321474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.330030 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 872/1758] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:49:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:49:50.331118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.333157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.348980 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 887/1759] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/projects => generated 401 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.348980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.464487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60662b9a-dac5-41c7-a28d-78cb63417a50 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.465036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073be76-bd3c-4c24-9496-aaf6282219db None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.470367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.471307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.471589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.471859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.472489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.474569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.506072 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 888/1760] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/users => generated 301 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.507475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.509398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.529369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21691ec9-5495-44eb-a134-26590a854b5f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.537650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.538674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.538956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.539703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] PATH_INFO: `/v3/projects/c316de3af02c4b9d996154c8db97ed1c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.540888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c316de3af02c4b9d996154c8db97ed1c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.543662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.569163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.571113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.571470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.571817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.574625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.593603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.598648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.616940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 873/1761] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:50.617802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.619691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c316de3af02c4b9d996154c8db97ed1c', 'request_id': 'req-4a475444-dd4a-4633-b3ca-a55e0a7804b4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:50.628133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c316de3af02c4b9d996154c8db97ed1c', 'request_id': 'req-4a475444-dd4a-4633-b3ca-a55e0a7804b4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:50.648910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c316de3af02c4b9d996154c8db97ed1c', 'request_id': 'req-4a475444-dd4a-4633-b3ca-a55e0a7804b4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:50.651020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928c5e3-133b-470c-8fcd-d1562d6f0ea3 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.682290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] The token cache is being invalidate because project c316de3af02c4b9d996154c8db97ed1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:50.686716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:50.689184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:50.692900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 889/1762] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:50 2022] DELETE /identity/v3/projects/c316de3af02c4b9d996154c8db97ed1c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:50.695211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.698045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.711037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:50.790269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:50.806730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.808333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.809067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.810212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.811021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.814613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9b6f-0702-452f-b55c-90d1a2895592 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.864084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a475444-dd4a-4633-b3ca-a55e0a7804b4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:50.874694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 874/1763] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.876534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3e3e-0031-4635-88bb-5c8769cc10da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.877357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:50.877770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3e3e-0031-4635-88bb-5c8769cc10da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.878682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3e3e-0031-4635-88bb-5c8769cc10da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.878989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:50.881179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:50.881592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:50.882380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:50.886214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:50.926291 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 890/1764] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/projects => generated 391 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:50.927357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:50.929331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:50.933897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-092b3e3e-0031-4635-88bb-5c8769cc10da None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:50.939241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-092b3e3e-0031-4635-88bb-5c8769cc10da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.028838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff2e00-f015-44f5-b9af-aed8f06ef9c3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.034858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.036699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.036699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.038125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.039011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.041607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.076303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 891/1765] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/users => generated 301 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:51.079910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.083936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.123373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 875/1766] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:49:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:51.130081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.130081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.130081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.159612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5acef1-b18a-42ba-97bf-0bf2904071fd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.177010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.177645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.178032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.178288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.178975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.180760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.182969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:51.184360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.218716 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 892/1767] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.218716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.222923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 876/1768] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:51.224102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.231490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.250920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb735c2-f071-4c68-8179-91b58ac397b4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.272385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.274012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.274012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.274012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.277066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.280855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6b45-8122-4cc0-8bc7-e0c6182afd6f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.294825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.302584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.303217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.303491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.303760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.304399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.305998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.342895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 893/1769] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.344731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.373163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2f81-f187-4582-9e2d-fa801be5b70e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.387978 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 877/1770] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:51 2022] POST /identity/v3/users => generated 302 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:51.393096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.393787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.394068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.394353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.395788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.396398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.400455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.402062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.418552 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 894/1771] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.419706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.437315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede15e0-f9d2-47d5-9ee1-0414107eec4a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.454592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.455636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.456062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.456489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] PATH_INFO: `/v3/projects/b2477085569a4f058d27169122d5208f/users/17d98450c23947fdb9a196397eb45c61/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.485646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2477085569a4f058d27169122d5208f, user_id=17d98450c23947fdb9a196397eb45c61, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.489811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d8bf28-1ea8-4c49-95af-e8c9d43838ee admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.516757 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 895/1772] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:51 2022] PUT /identity/v3/projects/b2477085569a4f058d27169122d5208f/users/17d98450c23947fdb9a196397eb45c61/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:51.519114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f677997-d241-45f4-8884-92f25efb1250 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.521622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.521916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.522192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.547990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:51.549884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:51.567379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:51.678602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.679236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 896/1773] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:51.685993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.687934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.688979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.732496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.746311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:51.765322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d5c9a-ad94-4256-b320-e704c4792f1b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.776349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.777095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.777472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.777891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.778547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.780221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.811683 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 897/1774] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.812795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.849772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48779340-cce6-490e-9e8d-9e8c83c27dbc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.876678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.878167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.880254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.884089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.884448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.884811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.887184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.889234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.925080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 878/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:51.926414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.933316 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 898/1776] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.934920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:51.954929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17763aa-aa32-4b5d-9a8d-aaa4ad8433ad admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.958461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd1d32-d62a-4145-9753-e23a60f69adf service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:51.960878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.961652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.962070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.962518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.963292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.966253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.973170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:51.973805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:51.974088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:51.974367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:51.975093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:51.976720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:51.991075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 899/1777] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:51.992440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.003076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 879/1778] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:52.006267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23eb54-5391-447e-8f6d-a84678d3c676 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.007701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.019261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777df879-7ca7-40a6-ada8-9f7ae589ec8c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.019825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.020800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.021143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.021568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.022174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:49:52.022539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.024096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.033395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.034094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.034432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.034760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] PATH_INFO: `/v3/projects/2cebe12de27243c5b9be5ed421ebdb6e/users/40bb8d1e3f184350a8dd178699af1f9b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.047980 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 900/1779] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:49:52.048980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.065661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1284a9-dfb9-4960-b68c-5dbc348408ea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.075852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.076538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.076817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.077091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] PATH_INFO: `/v3/projects/cc5279619b1545689c41e444f0171e65/users/837d2b775ce7490c8d7ba1a1db1e5c64/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.082529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cebe12de27243c5b9be5ed421ebdb6e, user_id=40bb8d1e3f184350a8dd178699af1f9b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.089094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.112690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5279619b1545689c41e444f0171e65, user_id=837d2b775ce7490c8d7ba1a1db1e5c64, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.118542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 880/1780] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:52 2022] PUT /identity/v3/projects/2cebe12de27243c5b9be5ed421ebdb6e/users/40bb8d1e3f184350a8dd178699af1f9b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:52.128832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.130304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.152550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 901/1781] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:52 2022] PUT /identity/v3/projects/cc5279619b1545689c41e444f0171e65/users/837d2b775ce7490c8d7ba1a1db1e5c64/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:52.152550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.212684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8bdb4-d2c7-4ce3-ad66-9f1773b8cce8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.236713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3a6be-a33e-4645-a8de-ee12f7ec71c7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.243570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.244261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.244544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.244958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] PATH_INFO: `/v3/projects/a3ff52bec20740639122631bf53608ed/users/c7ff45029cc248b49b1ebafca987c6f8/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.266259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ff52bec20740639122631bf53608ed, user_id=c7ff45029cc248b49b1ebafca987c6f8, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.269093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.290680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 902/1782] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:52 2022] PUT /identity/v3/projects/a3ff52bec20740639122631bf53608ed/users/c7ff45029cc248b49b1ebafca987c6f8/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:52.292388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.321557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.322224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.322670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.322957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] PATH_INFO: `/v3/projects/b2477085569a4f058d27169122d5208f/users/17d98450c23947fdb9a196397eb45c61/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.325204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2477085569a4f058d27169122d5208f, user_id=17d98450c23947fdb9a196397eb45c61, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.327934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437e73b-7d6f-4ef3-8123-1e011b7c2898 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.339809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:52.350594 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 881/1783] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:52 2022] PUT /identity/v3/projects/b2477085569a4f058d27169122d5208f/users/17d98450c23947fdb9a196397eb45c61/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:52.352282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.354056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.354673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.376444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6b93a-5831-4b46-aeed-ddc39dbaa946 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.405764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] MFA Rules not processed for user `40bb8d1e3f184350a8dd178699af1f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:52.407199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:52.417130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:52.494354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.494915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.495193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.495461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.496090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.498237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5777376-2195-43bd-b37e-a88fc8b1509d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.541663 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 882/1784] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:52.542741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.546732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 903/1785] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/projects => generated 409 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:52.553786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec4eaa-06e2-4fe0-9227-a9cbdc78ddaa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.554086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec4eaa-06e2-4fe0-9227-a9cbdc78ddaa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.554360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec4eaa-06e2-4fe0-9227-a9cbdc78ddaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.560990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e148ac5-0afe-4b66-a62e-62655fc409b0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.598560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-38ec4eaa-06e2-4fe0-9227-a9cbdc78ddaa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:52.599938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-38ec4eaa-06e2-4fe0-9227-a9cbdc78ddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:52.689432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.690224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.690681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.691051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.691887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.694554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.724496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 904/1786] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:52.726069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.726360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.726656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.737365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 883/1787] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:52.738295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.758143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b1c83-1d34-4e7b-85b2-d3b748b6bdd8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:52.765172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:52.765875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:52.766180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:52.766426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:52.767319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:52.769448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:52.784215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] MFA Rules not processed for user `17d98450c23947fdb9a196397eb45c61`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:52.785529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:52.842445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 884/1788] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:52.843501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.923843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:52.961230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 905/1789] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:52.962710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:52.990083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a727a6a-d1a7-4a93-8837-f0fd057e234a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.004334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.004996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.005301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.005578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.006270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.008206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.047324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 885/1790] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:52 2022] POST /identity/v3/users => generated 305 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:53.048364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.061478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019dd71-cc41-4544-adcb-d875400e993b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.069935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.070926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.071340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.071627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] PATH_INFO: `/v3/projects/cc5279619b1545689c41e444f0171e65/users/837d2b775ce7490c8d7ba1a1db1e5c64/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.073137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aeffda-06cf-4bcf-9053-1140f70eb32a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.074421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5279619b1545689c41e444f0171e65, user_id=837d2b775ce7490c8d7ba1a1db1e5c64, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.078983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.084067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:53.098456 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 886/1791] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:53 2022] PUT /identity/v3/projects/cc5279619b1545689c41e444f0171e65/users/837d2b775ce7490c8d7ba1a1db1e5c64/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:53.099412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.241015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.244507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.244507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.244507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.256171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b46b41-2435-467e-86aa-ec8c8563d74b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.266816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.271104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5410281-759c-44e9-bd2d-787d98d99cb2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.282962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.296337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 906/1792] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:53.300327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.300696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.301073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.302927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.303354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.309367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.313921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.320230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.347666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] MFA Rules not processed for user `50d8a91794144d73a8942f5c3bac8cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:53.348980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:53.414190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 887/1793] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:53 2022] POST /identity/v3/users => generated 298 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:53.416040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.455246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b9ef5-f07d-4414-8bdc-e59bc196c762 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.470784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.471559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.471943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.472337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.474189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.478565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc1ee-9020-4f17-b47e-c6b3c7525df1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.528501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 888/1794] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:53.533112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.533613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.534042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.554848 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 907/1795] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:53.558886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.606383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108126a-c85f-48fa-b977-0a030c1cc668 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.631289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.631639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.631841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.637051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.638129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.639822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.652697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] MFA Rules not processed for user `837d2b775ce7490c8d7ba1a1db1e5c64`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:53.653698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:53.683778 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 908/1796] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:53.694014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.738100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7160ee9-8283-4f37-acc4-b406034f974a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.763525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.766994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.767625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.767625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.768203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.774419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.777424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 889/1797] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:53.777851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.780198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:53.804401 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 909/1798] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:53.805693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.839595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845f038-c244-4372-9ee9-acb7dcb11eac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:53.851276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:53.853467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:53.855734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:53.856119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] PATH_INFO: `/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5/users/d096ab3e1b3e429684c203eaeaa35d80/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:53.913490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af03f41fb0b41cfad8d1f2789f985c5, user_id=d096ab3e1b3e429684c203eaeaa35d80, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:53.916412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:53.934325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:53.937896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 910/1799] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:53 2022] PUT /identity/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5/users/d096ab3e1b3e429684c203eaeaa35d80/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:53.938446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:53.963385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:53.970178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1cf0b-0346-4322-97e1-355737c7d021 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.075142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.075800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.076311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.076311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.083235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:54.083813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbe679-948c-4644-9125-61a1ddd63a2d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.096947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.098418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.098895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.099231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] PATH_INFO: `/v3/projects/acef8993ec2043f09bc31103aaa55c26/users/f10d9ba7b6dd477ba7f359a3c33938fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.146356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=acef8993ec2043f09bc31103aaa55c26, user_id=f10d9ba7b6dd477ba7f359a3c33938fc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:54.151461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e6573-14fd-4e53-a657-afea3978f0ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:54.185473 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 911/1800] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:53 2022] PUT /identity/v3/projects/acef8993ec2043f09bc31103aaa55c26/users/f10d9ba7b6dd477ba7f359a3c33938fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:54.189738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391798-e8bc-4bb6-b61a-2ac1a034e828 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.190159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391798-e8bc-4bb6-b61a-2ac1a034e828 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.192643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d391798-e8bc-4bb6-b61a-2ac1a034e828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.262697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:54.264967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:54.294115 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 890/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:54.295110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:54.316005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0d391798-e8bc-4bb6-b61a-2ac1a034e828 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:54.318468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d391798-e8bc-4bb6-b61a-2ac1a034e828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:54.338780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6815db9-4a8d-4c4e-b8ea-63238164f25e service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.470304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.471153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.472434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.473616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.474685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:54.482217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:54.510915 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 891/1802] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:54.511988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:54.518237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:54.535818 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 912/1803] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:54.537114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.537406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.539617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.598442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] MFA Rules not processed for user `f10d9ba7b6dd477ba7f359a3c33938fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:54.601754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:54.602021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:54.686080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c840042-b42c-402b-be36-1f01612f89cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.698306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.699179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.699483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.700062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.701843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:54.720357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 913/1804] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:54 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:54.721731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:54.738250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb588bd-1659-4cf0-a864-b889c3096072 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.745329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.746554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.748659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.750474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] PATH_INFO: `/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5/users/d096ab3e1b3e429684c203eaeaa35d80/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:54.758797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af03f41fb0b41cfad8d1f2789f985c5, user_id=d096ab3e1b3e429684c203eaeaa35d80, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:54.769049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:54.796655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 914/1805] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:54 2022] PUT /identity/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5/users/d096ab3e1b3e429684c203eaeaa35d80/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:54.799386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:54.801706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:54.805834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:54.808269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:54.832527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 892/1806] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:54.835380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:54.861096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1be00-94e1-4f28-8cba-2d9c6a1e7935 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.980397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d37e-dc7d-49c0-8377-fd6790b37273 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:54.995271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943264e2-6d26-499c-888f-84e94def599d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:54.997074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943264e2-6d26-499c-888f-84e94def599d service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:54.997440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943264e2-6d26-499c-888f-84e94def599d service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:54.999143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943264e2-6d26-499c-888f-84e94def599d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.005300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:55.006409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:55.006715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:55.007507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.010303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:55.012617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd7aeb-b16c-4caa-a468-261e2e7dd0eb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:55.076014 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 893/1807] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:49:54 2022] POST /identity/v3/projects => generated 411 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:55.090768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:55.090768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:55.091065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.138751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943264e2-6d26-499c-888f-84e94def599d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:55.140158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943264e2-6d26-499c-888f-84e94def599d service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:55.154387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] MFA Rules not processed for user `d096ab3e1b3e429684c203eaeaa35d80`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:55.155671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:55.162744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 915/1808] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:49:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:49:55.163594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943264e2-6d26-499c-888f-84e94def599d service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:55.192281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943264e2-6d26-499c-888f-84e94def599d service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:55.368060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:55.569174 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 894/1809] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:55.622645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:55.623202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:55.623444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:55.623722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.624378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:55.644415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:55.673239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:55.699928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9a2cc-4944-4ded-87f5-b770640f98ef None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:55.702620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:55.715982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:55.724216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:55.725293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:55.725780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] PATH_INFO: `/v3/users/9f3c11be6cc84332b079da481c6820b7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.729767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3c11be6cc84332b079da481c6820b7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:55.733056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:55.747854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 916/1810] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:49:55 2022] POST /identity/v3/users => generated 312 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:49:55.751550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:55.837515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c04c7-0f65-42ba-a10b-d07ce51a30aa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:55.876209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:55.881641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:55.883349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:55.883673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:55.884970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:55.892229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:55.893577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3c11be6cc84332b079da481c6820b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:55.898665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3c11be6cc84332b079da481c6820b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:55.915162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3c11be6cc84332b079da481c6820b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:55.944586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3c11be6cc84332b079da481c6820b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:55.949669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc9d97b1-c610-4f71-9cc4-c8a0f1d00b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3c11be6cc84332b079da481c6820b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:55.957645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 917/1811] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:55.958484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:55.972077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.027123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbbfd3-8c9b-4741-b8fa-c1a87b374453 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:56.046132 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 895/1812] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:55 2022] DELETE /identity/v3/users/9f3c11be6cc84332b079da481c6820b7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:56.048355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.048647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.048929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.059038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:56.059691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.059973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.060248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] PATH_INFO: `/v3/projects/08c5c7add4b946be9f61ffaf4badb33e/users/e8b307de1eea436b85505a55d8938f3e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.116806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:56.118996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.134908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c5c7add4b946be9f61ffaf4badb33e, user_id=e8b307de1eea436b85505a55d8938f3e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:56.143770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:56.194797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 918/1813] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:55 2022] PUT /identity/v3/projects/08c5c7add4b946be9f61ffaf4badb33e/users/e8b307de1eea436b85505a55d8938f3e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:56.196938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:56.259692 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 896/1814] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:56.335175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:56.336636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.444089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d9d21-7f58-4200-9e31-12d3014e1ad1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:56.465024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:56.468428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.470091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.471830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.474088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:56.475054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:56.498099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:56.506942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd968-5ff9-4d50-81b9-90ecf626ed4c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:56.539032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 919/1815] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:49:56.540075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:56.548514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.725238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:56.726327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.726789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.727238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.756416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:56.757997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:56.780722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0abe5a-8bf4-46c6-a59b-ef952119c26d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:56.782402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 897/1816] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:49:56.783735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:56.794342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:56.795179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.795868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.796000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.798942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701fb5c-0d9d-46df-b848-7147372c1590 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:56.805584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.818797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:56.819804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.820244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.820686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] PATH_INFO: `/v3/projects/08c5c7add4b946be9f61ffaf4badb33e/users/e8b307de1eea436b85505a55d8938f3e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.824509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c5c7add4b946be9f61ffaf4badb33e, user_id=e8b307de1eea436b85505a55d8938f3e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:56.828826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8484b-f449-4a39-b0cf-429f14df9a5a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:56.850615 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 898/1817] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:49:56 2022] PUT /identity/v3/projects/08c5c7add4b946be9f61ffaf4badb33e/users/e8b307de1eea436b85505a55d8938f3e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:56.866724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:56.866724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:56.866724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:56.930032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] MFA Rules not processed for user `e8b307de1eea436b85505a55d8938f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:56.933059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:56.955147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:56.980260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:56.981889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:57.000311 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 920/1818] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:57.032541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 899/1819] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:56 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:57.050830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:57.053135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.064114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:57.067913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.082606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84253b-def1-4356-bc6c-6e8b2ebebaa4 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:57.261990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cf52b-f379-46d6-b1c8-082c8d0243c8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:57.268707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:57.269459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:57.269740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:57.271428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:57.272000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:57.272965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:57.273497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:57.286714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] PATH_INFO: `/v3/users/40bb8d1e3f184350a8dd178699af1f9b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:57.289515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40bb8d1e3f184350a8dd178699af1f9b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:57.300559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:57.304833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:57.308941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572a988-9c1e-4fdf-93ae-64923fa51051 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:57.338657 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 921/1820] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:57.342055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92009-4124-4760-b63d-a7edea3f6328 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:57.342343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92009-4124-4760-b63d-a7edea3f6328 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:57.343334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c92009-4124-4760-b63d-a7edea3f6328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:57.358689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:57.429106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40bb8d1e3f184350a8dd178699af1f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:57.431303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40bb8d1e3f184350a8dd178699af1f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:57.439804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b6c92009-4124-4760-b63d-a7edea3f6328 None None] MFA Rules not processed for user `0686ece5b3ea46269d1098999ba98330`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:57.444537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6c92009-4124-4760-b63d-a7edea3f6328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.449176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40bb8d1e3f184350a8dd178699af1f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:57.471784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40bb8d1e3f184350a8dd178699af1f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:57.473364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40bb8d1e3f184350a8dd178699af1f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:57.497763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 900/1821] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:57 2022] DELETE /identity/v3/users/40bb8d1e3f184350a8dd178699af1f9b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:57.498664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:57.505898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.799818 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 922/1822] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:57.804435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d493-c75b-48ee-bab6-2d51609b32ad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:57.804746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d493-c75b-48ee-bab6-2d51609b32ad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:57.805767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566d493-c75b-48ee-bab6-2d51609b32ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:57.861336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6566d493-c75b-48ee-bab6-2d51609b32ad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:57.862706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6566d493-c75b-48ee-bab6-2d51609b32ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.870150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cb7ab-8a54-4c8b-8c49-013bcb08c49d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:57.900712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:57.901345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:57.901663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:57.903913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:57.930467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:57.994638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:58.007349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 923/1823] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:58.008802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.009340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.010576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.089772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:58.092066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:58.159540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.161633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.195270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 901/1824] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:58.196251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.198323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:58.221402 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 924/1825] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 24 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:58.236584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.247294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269d16-a7b2-44fc-a2ef-d70898350258 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.258162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:58.258832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.259116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.259402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.264959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f308fa-bb85-41b0-b7df-690334a7d320 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.272552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:58.273234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.273641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.273843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.282952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.285227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:58.287431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.325414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 902/1826] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:58.326613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.348242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed6288-9f08-4e46-82eb-49fee90791c7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.354690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:58.355306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.357418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.359180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.381165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.382767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.414612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 903/1827] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:58.443807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.445544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.460826 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 925/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:49:58.516946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.519801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:58.523874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.543541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c03d89-98c5-4434-8e1a-21cccca447b0 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.561052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ea018-86e4-44be-ab6c-6da96adfba44 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.562756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:58.563745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.566014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.566683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.596995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:58.597722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.598012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.598462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] PATH_INFO: `/v3/users/f10d9ba7b6dd477ba7f359a3c33938fc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.600198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10d9ba7b6dd477ba7f359a3c33938fc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.604258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.607390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:58.774637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10d9ba7b6dd477ba7f359a3c33938fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:58.775418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:58.776924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10d9ba7b6dd477ba7f359a3c33938fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:58.777384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:58.793274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10d9ba7b6dd477ba7f359a3c33938fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:58.809858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10d9ba7b6dd477ba7f359a3c33938fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:58.811719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a974960-6c54-41ea-99c5-18d652c0817c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10d9ba7b6dd477ba7f359a3c33938fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:58.853569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 926/1830] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:49:58.854657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:58.860175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:58.860754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 904/1830] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:58 2022] DELETE /identity/v3/users/f10d9ba7b6dd477ba7f359a3c33938fc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:58.865167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:58.866443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:58.868867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:58.913740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77836534-0865-400c-8d60-ad1669b55613 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:58.961458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:49:58.965351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:59.010282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:59.159559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:59.160228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:59.160510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:59.160781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] PATH_INFO: `/v3/projects/03b8651c3c7d4ae6a200f8613a96c840` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:59.169567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b8651c3c7d4ae6a200f8613a96c840)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:59.171856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:59.207736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 905/1831] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:49:59.232644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:59.239485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:59.289052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b8651c3c7d4ae6a200f8613a96c840', 'request_id': 'req-34500f64-e5e6-4590-aa39-10b7bd16d811'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.297150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b8651c3c7d4ae6a200f8613a96c840', 'request_id': 'req-34500f64-e5e6-4590-aa39-10b7bd16d811'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.313413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b8651c3c7d4ae6a200f8613a96c840', 'request_id': 'req-34500f64-e5e6-4590-aa39-10b7bd16d811'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.338183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] The token cache is being invalidate because project 03b8651c3c7d4ae6a200f8613a96c840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:49:59.338587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.346993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.349225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 927/1832] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:49:58 2022] DELETE /identity/v3/projects/03b8651c3c7d4ae6a200f8613a96c840 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:49:59.353306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:49:59.354617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:59.418118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7ef34-a24e-4623-a240-a148cf2b6f3b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:59.512848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500f64-e5e6-4590-aa39-10b7bd16d811 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:49:59.531134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:59.531861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:59.532144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:59.532421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] PATH_INFO: `/v3/users/837d2b775ce7490c8d7ba1a1db1e5c64` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:59.533669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837d2b775ce7490c8d7ba1a1db1e5c64)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:59.536218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:59.580375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:59.653062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:49:59.655106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:49:59.657782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:49:59.659026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:49:59.665527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:49:59.708553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:49:59.709980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f0b80-d9dc-4f3f-a5b7-16e1a6d456c2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:49:59.726913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837d2b775ce7490c8d7ba1a1db1e5c64'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.730064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837d2b775ce7490c8d7ba1a1db1e5c64'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.752428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837d2b775ce7490c8d7ba1a1db1e5c64'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.758746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 906/1833] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:49:59.785941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:49:59.819788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837d2b775ce7490c8d7ba1a1db1e5c64'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.820615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837d2b775ce7490c8d7ba1a1db1e5c64'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:49:59.858785 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 928/1834] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:49:59 2022] DELETE /identity/v3/users/837d2b775ce7490c8d7ba1a1db1e5c64 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:00.087881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:00.088242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:00.088434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:00.119571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:00.120878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:00.286362 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 907/1835] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:50:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:00.293814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:00.297987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:00.504416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ea7cd-c61e-4e12-b882-6e38673e5553 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:00.510559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:00.511455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:00.512171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:00.512881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:00.530826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:00.533457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:00.564886 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 929/1836] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:50:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:50:01.607142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:01.609053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:01.801015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ce80c-77f0-44dd-9cf1-f534cafd3ef8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:01.816952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:01.823164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:01.823475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:01.826907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] PATH_INFO: `/v3/projects/2cebe12de27243c5b9be5ed421ebdb6e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:01.829982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cebe12de27243c5b9be5ed421ebdb6e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:01.835804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:01.985300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cebe12de27243c5b9be5ed421ebdb6e', 'request_id': 'req-5ad1e4b5-01b4-453d-94db-0d2881f0f799'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.005044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cebe12de27243c5b9be5ed421ebdb6e', 'request_id': 'req-5ad1e4b5-01b4-453d-94db-0d2881f0f799'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.022526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cebe12de27243c5b9be5ed421ebdb6e', 'request_id': 'req-5ad1e4b5-01b4-453d-94db-0d2881f0f799'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.054033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] The token cache is being invalidate because project 2cebe12de27243c5b9be5ed421ebdb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:02.056036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.057979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.071210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:02.071938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 908/1837] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:01 2022] DELETE /identity/v3/projects/2cebe12de27243c5b9be5ed421ebdb6e => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:02.090605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026d27-270e-4704-bbc5-6348fe0220d4 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:02.270529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:02.273376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:02.309822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:02.319947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:02.320610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:02.320895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:02.321172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] PATH_INFO: `/v3/users/d096ab3e1b3e429684c203eaeaa35d80` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:02.335103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d096ab3e1b3e429684c203eaeaa35d80)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:02.341017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:02.461926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd096ab3e1b3e429684c203eaeaa35d80'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.466409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd096ab3e1b3e429684c203eaeaa35d80'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.487668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd096ab3e1b3e429684c203eaeaa35d80'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.527652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1e4b5-01b4-453d-94db-0d2881f0f799 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:02.540825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:02.543319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:02.543864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:02.544147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:02.544461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] PATH_INFO: `/v3/projects/acef8993ec2043f09bc31103aaa55c26` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:02.545420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd096ab3e1b3e429684c203eaeaa35d80'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.545638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acef8993ec2043f09bc31103aaa55c26)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:02.548300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:02.550162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd096ab3e1b3e429684c203eaeaa35d80'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.653439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 930/1838] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:02 2022] DELETE /identity/v3/users/d096ab3e1b3e429684c203eaeaa35d80 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:02.670169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acef8993ec2043f09bc31103aaa55c26', 'request_id': 'req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.682764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acef8993ec2043f09bc31103aaa55c26', 'request_id': 'req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.707353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acef8993ec2043f09bc31103aaa55c26', 'request_id': 'req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.737596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] The token cache is being invalidate because project acef8993ec2043f09bc31103aaa55c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:02.738104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.739536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:02.743350 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 909/1839] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:02 2022] DELETE /identity/v3/projects/acef8993ec2043f09bc31103aaa55c26 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:02.747852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:02.750874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:03.047190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd128b8f-3c59-4fc0-8d95-ba7e8790f7c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:03.065310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:03.065970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:03.066318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:03.072739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] PATH_INFO: `/v3/users/e8b307de1eea436b85505a55d8938f3e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:03.075473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b307de1eea436b85505a55d8938f3e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:03.086005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:03.267236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b307de1eea436b85505a55d8938f3e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:03.271915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b307de1eea436b85505a55d8938f3e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:03.301837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b307de1eea436b85505a55d8938f3e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:03.349012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b307de1eea436b85505a55d8938f3e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:03.351044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b307de1eea436b85505a55d8938f3e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:03.415262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 931/1840] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:02 2022] DELETE /identity/v3/users/e8b307de1eea436b85505a55d8938f3e => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:04.445215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:04.446811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:04.808441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4026a-834d-4f9a-ba86-fcc32cddddc2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:04.958553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:04.960094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:05.049628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:05.101359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:05.106467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.106467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.110892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] PATH_INFO: `/v3/projects/cc5279619b1545689c41e444f0171e65` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.126104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5279619b1545689c41e444f0171e65)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:05.128958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:05.189840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5def5-e02a-42fc-94a1-cbe9bd31b333 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:05.212354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:05.213588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.214121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.214612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] PATH_INFO: `/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.215974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af03f41fb0b41cfad8d1f2789f985c5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:05.222716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:05.246584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:05.366546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af03f41fb0b41cfad8d1f2789f985c5', 'request_id': 'req-df180785-45f7-4302-a3cd-f9dde20f4a57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.389162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af03f41fb0b41cfad8d1f2789f985c5', 'request_id': 'req-df180785-45f7-4302-a3cd-f9dde20f4a57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.403650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af03f41fb0b41cfad8d1f2789f985c5', 'request_id': 'req-df180785-45f7-4302-a3cd-f9dde20f4a57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.428786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5279619b1545689c41e444f0171e65', 'request_id': 'req-792302b3-44e5-4d83-a54c-24107ea25b35'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.436027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] The token cache is being invalidate because project 7af03f41fb0b41cfad8d1f2789f985c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:05.440599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.442863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df180785-45f7-4302-a3cd-f9dde20f4a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.452184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 932/1841] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:04 2022] DELETE /identity/v3/projects/7af03f41fb0b41cfad8d1f2789f985c5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:05.453642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9097-802a-4132-881b-ec9da61880d2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.453642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9097-802a-4132-881b-ec9da61880d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.457245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d9097-802a-4132-881b-ec9da61880d2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.465006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5279619b1545689c41e444f0171e65', 'request_id': 'req-792302b3-44e5-4d83-a54c-24107ea25b35'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.465689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 933/1842] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:05 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:05.467156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7879c9-1572-44bb-bcb9-701a1362b355 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.467478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7879c9-1572-44bb-bcb9-701a1362b355 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.467738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7879c9-1572-44bb-bcb9-701a1362b355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.490946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5279619b1545689c41e444f0171e65', 'request_id': 'req-792302b3-44e5-4d83-a54c-24107ea25b35'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.537240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] The token cache is being invalidate because project cc5279619b1545689c41e444f0171e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:05.537978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.540571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-792302b3-44e5-4d83-a54c-24107ea25b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:05.545365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 910/1843] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:04 2022] DELETE /identity/v3/projects/cc5279619b1545689c41e444f0171e65 => generated 0 bytes in 1101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:05.548250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fd7879c9-1572-44bb-bcb9-701a1362b355 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:05.550840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd7879c9-1572-44bb-bcb9-701a1362b355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:05.551675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e98189-02a0-4f6e-a44b-e5fa2829b9d9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.552116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e98189-02a0-4f6e-a44b-e5fa2829b9d9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.552373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e98189-02a0-4f6e-a44b-e5fa2829b9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.609801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-99e98189-02a0-4f6e-a44b-e5fa2829b9d9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:05.612349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99e98189-02a0-4f6e-a44b-e5fa2829b9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:05.751972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 911/1844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:05.754301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.755408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.755408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.789275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:05.790514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:05.820985 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 934/1845] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:05.823303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34563492-6544-47db-b256-f3fb587d7243 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.823928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34563492-6544-47db-b256-f3fb587d7243 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.824428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34563492-6544-47db-b256-f3fb587d7243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.892694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-34563492-6544-47db-b256-f3fb587d7243 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:05.894384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34563492-6544-47db-b256-f3fb587d7243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:05.907188 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 935/1846] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:05.909074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:05.909739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:05.910220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:05.973005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:05.980924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:06.019604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 936/1847] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:06.028525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.035932 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 912/1848] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:06.036925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.081466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8bed-b7e9-4975-ac03-4c4aee24dd3e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.082271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a870a1-e3af-4340-8e1d-db422a364507 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.106489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.107222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.107411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.107918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.108032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.108221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.109052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.109822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:06.110317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.110647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.113124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:06.113307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:06.113307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.119431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:06.128127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:06.151542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 913/1849] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:06.152373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.180410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 937/1850] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:06.180659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b02c3-9e22-4758-ae53-e0274d5fd708 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.181582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.205180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.205882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.206166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.206442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.207142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.210016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:06.219268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4d080-7434-4df8-a1ec-34942c7ebcf4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.241693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.247236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.251987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.255057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.256707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.267506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:06.319404 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 914/1851] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:06 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:06.320435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.322644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:06.363059 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 938/1852] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:06 2022] POST /identity/v3/projects => generated 399 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:06.375365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.501791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67350bd0-0fa5-4a67-93e9-baec1db6ad1d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.560920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:06.608923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a17c7-635d-47fc-b040-91248130e54f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:06.620927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.621591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.621866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.622140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.622810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.625637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:06.733496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 939/1853] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:06 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:06.738235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:06.749457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:06.779376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:06.780156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:06.781136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:06.781437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] PATH_INFO: `/v3/projects/08c5c7add4b946be9f61ffaf4badb33e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:06.787247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c5c7add4b946be9f61ffaf4badb33e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:06.789290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.008836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c5c7add4b946be9f61ffaf4badb33e', 'request_id': 'req-a2d54ff0-039e-4c29-a54a-2738bd84d731'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:07.022583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c5c7add4b946be9f61ffaf4badb33e', 'request_id': 'req-a2d54ff0-039e-4c29-a54a-2738bd84d731'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:07.037907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c5c7add4b946be9f61ffaf4badb33e', 'request_id': 'req-a2d54ff0-039e-4c29-a54a-2738bd84d731'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:07.072344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] The token cache is being invalidate because project 08c5c7add4b946be9f61ffaf4badb33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:07.072666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:07.079414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:07.085483 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 915/1854] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:06 2022] DELETE /identity/v3/projects/08c5c7add4b946be9f61ffaf4badb33e => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:07.086816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.090540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.111452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a7e10-52f5-428b-97cb-1241b1494747 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.155467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:07.317286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d54ff0-039e-4c29-a54a-2738bd84d731 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.339369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.340274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.341629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.341900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.342340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.344893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.345488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.345936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.347720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.349414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.351777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.414617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.425248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8b22f-317e-4762-9671-485d54dfbec6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.459751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 916/1855] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:07 2022] POST /identity/v3/users => generated 300 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:07.461261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.461655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 940/1856] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:07.465564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.466203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.467879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.469338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.505441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:07.507274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.527854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b995d1-3b27-4033-9453-821e5b75b99f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.551196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.553759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.554057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.555629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.560556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.562433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922a46b-ce9c-4dc1-8641-c0ecaf5a0b8e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.629286 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 917/1857] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:07.630953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.631072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.631338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.642429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 941/1858] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 24 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:07.644719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.660972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c9a38-4f8f-441d-9829-5cdea9f47bf7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.712700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.715764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.716082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.717017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.735316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.735870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:07.735870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.736301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.749628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:07.769192 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 942/1859] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:07.772641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.781317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 918/1860] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:07.790340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.792752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:07.806719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d01c9d-c084-4b67-9ba8-eded47b3f81e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.823119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.824056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.824595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.825061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.825883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.828422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.845763 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 943/1861] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:07.846996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.854291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e631ce8-9b0a-47d5-a608-2da021f8cff5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.877949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762620b-c14c-4727-bb0a-45e20262ef5d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:07.884123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.885380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.886196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.887053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.891603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:07.893122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.894210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.895823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:07.897350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.902718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9bd9-8d5f-4c06-80d5-fc1e36bb323e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.911059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:07.917746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:07.943096 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 944/1862] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:07.962124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:07.963530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 919/1863] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:07.965075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:07.965764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:07.966926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:08.043545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52692-b30c-40b4-a418-e97cc2303920 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:08.084181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:08.088393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:08.090368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:08.093902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] PATH_INFO: `/v3/projects/349d3aa8627644d59867a744695f2579/users/93982876b1e24efb8ffa8c009532343a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:08.106051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:08.111013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:08.136711 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 945/1864] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:08.137665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:08.142967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349d3aa8627644d59867a744695f2579, user_id=93982876b1e24efb8ffa8c009532343a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:08.148300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:08.217570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5a24f-7f41-43c0-96ab-30b7496e565a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:08.251729 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 920/1865] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:07 2022] PUT /identity/v3/projects/349d3aa8627644d59867a744695f2579/users/93982876b1e24efb8ffa8c009532343a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:08.252992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:08.526956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:08.527671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:08.528108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:08.528210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] PATH_INFO: `/v3/projects/82024a37b59b4790b869f387793fd5bc/users/d8eebdb35d754b39b1b551a25e5195cb/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:08.557786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd137f6-0b0c-4396-be0d-d42f703de5e4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:08.585527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:08.591108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:08.591418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:08.591698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:08.592488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:08.592867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:08.598365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=82024a37b59b4790b869f387793fd5bc, user_id=d8eebdb35d754b39b1b551a25e5195cb, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:08.604305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb45ad3-8330-48aa-a210-6ee485d664bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:08.606754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:08.676561 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 946/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:08 2022] PUT /identity/v3/projects/82024a37b59b4790b869f387793fd5bc/users/d8eebdb35d754b39b1b551a25e5195cb/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:08.678242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b8a8a-621d-4b5e-8ca9-6164d8d80221 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:08.678729 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 921/1867] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:08.679015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b8a8a-621d-4b5e-8ca9-6164d8d80221 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:08.679308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b8a8a-621d-4b5e-8ca9-6164d8d80221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:08.679772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:08.723204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3636374-f1bb-49b8-b6df-50e092a5a87f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:08.746258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-285b8a8a-621d-4b5e-8ca9-6164d8d80221 None None] MFA Rules not processed for user `93982876b1e24efb8ffa8c009532343a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:08.747921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-285b8a8a-621d-4b5e-8ca9-6164d8d80221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:09.032408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:09.041428 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 947/1868] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:09.044051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:09.044051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:09.044051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:09.140826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:09.141413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:09.141698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:09.141976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:09.142654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:09.172817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:09.173955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] MFA Rules not processed for user `d8eebdb35d754b39b1b551a25e5195cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:09.182770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:09.295486 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 922/1869] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:08 2022] POST /identity/v3/projects => generated 387 bytes in 616 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:09.297218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:09.463021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:09.506008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e3dbf-3b5a-46a0-8688-90ca1ef30e5a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:09.520716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:09.521351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:09.521843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:09.522122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:09.522807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:09.527539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:09.598850 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 923/1870] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:09 2022] POST /identity/v3/projects => generated 371 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:09.599751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:09.646833 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 948/1871] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 605 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:09.650359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:09.711402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca01391-acc1-4e2e-991d-a6e999eff42d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:09.759175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130424be-b93f-4b79-aefe-8fca2fda79ec admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:09.769497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:09.770892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:09.771334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:09.771774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:09.772796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:09.784924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:09.863436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 924/1872] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:09 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:09.864804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:09.919169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76f17b-38a4-4ace-b44b-83c2a5f28be8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:09.945464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:09.946811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:09.947523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:09.949841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:09.953154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:09.964124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.062169 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 925/1873] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:09 2022] POST /identity/v3/projects => generated 399 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:10.063336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.149906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:10.224268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.240123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.240439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.240686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.241366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.248318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.323894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f643d-db43-448b-a804-796ed61f1c1d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.334007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 949/1874] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:09 2022] POST /identity/v3/users => generated 292 bytes in 686 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:10.343130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.361087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.377840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.378191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.378469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.391992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.398720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd47daa-69f1-4e29-880f-93770871c190 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.404494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.433145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.433813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.434325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.434557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.435352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.471421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.476832 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 926/1875] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:10.479906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.549099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec0711-4ff2-4d96-b28c-890d6ad047f0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.569733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.570597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.571017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.571422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.572252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.574182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.595697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 927/1876] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:10.596664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.606187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 950/1877] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:10 2022] POST /identity/v3/users => generated 306 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:10.609407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c3208-022d-4b7f-900e-0f006a87a0c7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.614798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.615480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.615996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.616336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.616449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.617194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.619950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.647181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb09f4-a843-4cac-8650-1d790f61804a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.648671 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 928/1878] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:10.650670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.664996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473cab-6d95-4779-847e-d2dc9b5fc02d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.674807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.675847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.676332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.676754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.677608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.680935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.680935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.681271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.681900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] PATH_INFO: `/v3/projects/00041f889f3d430d87831f82a4d3b598/users/4349ca208839405199602d34e1c7bf33/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.688739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.720548 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 929/1879] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:10.721678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.723714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] RBAC: Authorizing `identity:create_grant(project_id=00041f889f3d430d87831f82a4d3b598, user_id=4349ca208839405199602d34e1c7bf33, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:10.728565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:10.751393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265da9-b5b8-465f-8478-b9a1876860b1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.754422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 951/1880] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:10 2022] PUT /identity/v3/projects/00041f889f3d430d87831f82a4d3b598/users/4349ca208839405199602d34e1c7bf33/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:10.758441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:10.925443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c83f37-95a7-4ff7-85fa-896e584ebcab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:10.960429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.963232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.963746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.964259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] PATH_INFO: `/v3/projects/38a02cd040484a8c98525fd9cc37d266/users/9397b23c06f14ac396c4ecb5e41231dc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:10.977568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:10.978948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:10.979970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:10.982197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] PATH_INFO: `/v3/projects/0a84d3ddc23746749c1395d42e3d987d/users/2691f882a7c74fe9b29a8675ccb554c4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.001646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a02cd040484a8c98525fd9cc37d266, user_id=9397b23c06f14ac396c4ecb5e41231dc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.004790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.031348 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 952/1881] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:10 2022] PUT /identity/v3/projects/38a02cd040484a8c98525fd9cc37d266/users/9397b23c06f14ac396c4ecb5e41231dc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:11.032781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:11.035822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a84d3ddc23746749c1395d42e3d987d, user_id=2691f882a7c74fe9b29a8675ccb554c4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.042181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.051371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:11.075326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 930/1882] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:10 2022] PUT /identity/v3/projects/0a84d3ddc23746749c1395d42e3d987d/users/2691f882a7c74fe9b29a8675ccb554c4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:11.076259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:11.223550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231d216-a2ce-424d-8296-9e389529f06d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:11.275072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d6ab-f057-4775-b049-eb0c8f22ebb3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:11.288811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:11.289413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.289773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.291892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.293813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.296529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794815d-9252-42f0-ae7d-f43e0265b8a5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.326254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 931/1883] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:11.330125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18624-d348-4e33-bb71-f3380d51c068 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.330449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18624-d348-4e33-bb71-f3380d51c068 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.330802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18624-d348-4e33-bb71-f3380d51c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.391498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:11.424213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-07d18624-d348-4e33-bb71-f3380d51c068 None None] MFA Rules not processed for user `2691f882a7c74fe9b29a8675ccb554c4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:11.426788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07d18624-d348-4e33-bb71-f3380d51c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:11.431260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:11.433039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.437456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.438109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.439262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.441446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.494028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 953/1884] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:11.495248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:11.521311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008d085-3223-4376-b04b-27919a2c0c25 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:11.526895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:11.528040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.528424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.528802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] PATH_INFO: `/v3/projects/38a02cd040484a8c98525fd9cc37d266/users/9397b23c06f14ac396c4ecb5e41231dc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.531466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a02cd040484a8c98525fd9cc37d266, user_id=9397b23c06f14ac396c4ecb5e41231dc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.534232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.567078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 954/1885] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:11 2022] PUT /identity/v3/projects/38a02cd040484a8c98525fd9cc37d266/users/9397b23c06f14ac396c4ecb5e41231dc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:11.568445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:11.668266 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 932/1886] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:11.671341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00ed8-a9d0-4ec7-97eb-f3b5f57a1ce2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.671874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00ed8-a9d0-4ec7-97eb-f3b5f57a1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.672774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf00ed8-a9d0-4ec7-97eb-f3b5f57a1ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.743173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-acf00ed8-a9d0-4ec7-97eb-f3b5f57a1ce2 None None] MFA Rules not processed for user `9397b23c06f14ac396c4ecb5e41231dc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:11.744992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-acf00ed8-a9d0-4ec7-97eb-f3b5f57a1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:11.767464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d179a2-3db8-4a44-80ad-91e9e724b8a6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:11.784407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:11.785352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.785696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.786148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] PATH_INFO: `/v3/projects/00041f889f3d430d87831f82a4d3b598/users/4349ca208839405199602d34e1c7bf33/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:11.790455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00041f889f3d430d87831f82a4d3b598, user_id=4349ca208839405199602d34e1c7bf33, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:11.794051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:11.829166 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 955/1887] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:11 2022] PUT /identity/v3/projects/00041f889f3d430d87831f82a4d3b598/users/4349ca208839405199602d34e1c7bf33/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:11.830536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:11.913347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:11.954690 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 933/1888] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:11.956552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:11.957169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:11.957646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.006071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4d914-a14d-4959-84b1-50c67f23f03f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:12.023171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:12.023989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.024309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.024853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.026264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:12.028435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:12.109285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] MFA Rules not processed for user `4349ca208839405199602d34e1c7bf33`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:12.116663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 956/1889] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:11 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:12.116663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:12.118446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:12.135955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:12.420012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be326884-39ac-43d5-901a-22984b5c2a36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:12.438948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:12.439873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.440259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.440636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.445514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 934/1890] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:12.446574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:12.472262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:12.544144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:12.547403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8675eb-3a97-4404-aab4-d52ef04f8cc3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:12.565619 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 957/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:12.567146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3c74-817a-4b23-84fa-7187a489dab6 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.570119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3c74-817a-4b23-84fa-7187a489dab6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.570809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3c74-817a-4b23-84fa-7187a489dab6 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.575375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 958/1892] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:12.578903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fba51-9eaa-47d6-af26-cf00089753d8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.579297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fba51-9eaa-47d6-af26-cf00089753d8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.579998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fba51-9eaa-47d6-af26-cf00089753d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.653467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-427fba51-9eaa-47d6-af26-cf00089753d8 None None] MFA Rules not processed for user `93982876b1e24efb8ffa8c009532343a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:12.660528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-427fba51-9eaa-47d6-af26-cf00089753d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:12.668272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3e4fb-e564-4554-b617-cf81d299209e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:12.707892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:12.711625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.711819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.712308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.717133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:12.734089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:12.812129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:12.834546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 959/1893] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:12.835953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5a31-412e-4e5f-8b74-1baf48a19445 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.836243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5a31-412e-4e5f-8b74-1baf48a19445 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.836523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace5a31-412e-4e5f-8b74-1baf48a19445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.839921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 935/1894] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:12 2022] POST /identity/v3/users => generated 299 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:12.865325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:12.875606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7ace5a31-412e-4e5f-8b74-1baf48a19445 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:12.879892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7ace5a31-412e-4e5f-8b74-1baf48a19445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:12.908422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 960/1895] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:12.922552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34d1b5-6ef1-4d42-b62b-10853040fe31 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:12.953433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:12.957222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.968528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.968528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.983885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:12.985028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:12.985320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:12.985594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:12.989347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:13.024386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 936/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:13.025605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:13.031985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] MFA Rules not processed for user `d278977b201c48dda2d73823fbfe4833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:13.033580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:13.058939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653efbe7-f4d9-493f-8e83-7b3a4d6a2050 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:13.092192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:13.094025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:13.095311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:13.098567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] PATH_INFO: `/v3/projects/5af66b8e090845c58ea3e68fe1bd844a/users/24aee5bd1fff4471b4c537d76edd9a7e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:13.150288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af66b8e090845c58ea3e68fe1bd844a, user_id=24aee5bd1fff4471b4c537d76edd9a7e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:13.154317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:13.197199 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 961/1897] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:13.206539 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 937/1898] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:13 2022] PUT /identity/v3/projects/5af66b8e090845c58ea3e68fe1bd844a/users/24aee5bd1fff4471b4c537d76edd9a7e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:13.212381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:13.234940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367eeddb-aa18-4a7c-93a5-3861c82a5720 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:13.374271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:13.375032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:13.375321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:13.375596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:13.376265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:13.377947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356437e-bbd5-4b00-8330-5b281d23d7cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:13.408246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 962/1899] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:13.409478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:13.427197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11107632-8b43-446d-9d0d-be4c16ddd7d2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:13.436721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:13.439602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:13.439850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:13.440868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] PATH_INFO: `/v3/projects/5af66b8e090845c58ea3e68fe1bd844a/users/24aee5bd1fff4471b4c537d76edd9a7e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:13.446100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af66b8e090845c58ea3e68fe1bd844a, user_id=24aee5bd1fff4471b4c537d76edd9a7e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:13.449157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:13.479414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 938/1900] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:13 2022] PUT /identity/v3/projects/5af66b8e090845c58ea3e68fe1bd844a/users/24aee5bd1fff4471b4c537d76edd9a7e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:13.483505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:13.485278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:13.485610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:13.549830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] MFA Rules not processed for user `24aee5bd1fff4471b4c537d76edd9a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:13.552894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:13.707165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:13.741410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 963/1901] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:13.742421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:13.744303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:13.805692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:13.948983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d947ebc-0f95-418e-91c0-d5da2b864403 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:13.964219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:13.965815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:13.966422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:13.969337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:13.999764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0232c00-a964-42a8-8595-9322f300ea8e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:14.017283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:14.018045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:14.018324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:14.018626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:14.043057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:14.049116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f08d39-80d0-4830-b24d-618f9616439e service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:14.074406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:14.083921 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 964/1902] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:14.118100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:14.120429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:14.143095 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 939/1903] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:14.432866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:14.433159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:14.433440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:14.477241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] MFA Rules not processed for user `c7ff45029cc248b49b1ebafca987c6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:14.478694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:14.571443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:14.571666 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 965/1904] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:14 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:14.574108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:14.741111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a874-b2e7-4142-91a8-0c3814349dd7 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:14.758751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:14.759790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:14.760176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:14.760551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:14.761406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:14.772274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:14.824011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:14.842981 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 940/1905] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:14 2022] POST /identity/v3/projects => generated 403 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:14.847362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eeaadc-6077-4f9a-8937-d09ad439aebe None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:14.956016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:14.956992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:14.957323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:14.957703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:14.958420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:14.960640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:14.995816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 966/1906] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:14 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:15.000020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.016820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b119fa2-03fa-4a4b-80db-107078937662 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.028616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.029592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.030680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.031140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.035853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.037921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc310b5-05a7-4ff9-a514-de80b33d2ddd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.062467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.067584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 941/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:15.088746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34a902-f7d4-44c7-940a-e3689560bede admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.096047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.096340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.096621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.101103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.101974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.104028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.104906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] PATH_INFO: `/v3/projects/9249e1a798344c07ac7f828954a3a1bb/users/dd4ed3b4807c47f7a1771e7d01ba3f97/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.140214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:15.141785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:15.144567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9249e1a798344c07ac7f828954a3a1bb, user_id=dd4ed3b4807c47f7a1771e7d01ba3f97, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.148663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.153202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:15.194553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 967/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:15 2022] PUT /identity/v3/projects/9249e1a798344c07ac7f828954a3a1bb/users/dd4ed3b4807c47f7a1771e7d01ba3f97/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:15.195336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.258411 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 942/1909] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:50:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:15.259769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.285720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b24674-8cb3-49af-b8ad-e2e7f5ab8b49 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.366205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0dc7-c5a4-459c-9b99-d111d7141bb2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.382173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.382836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.383119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.383442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.432183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.433000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.433460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.433738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.434621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.436929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.462401 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 943/1910] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:15.463374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.466777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:15.475379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:15.515462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.517707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.546756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 968/1911] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:15.547897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.570580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb7c9c-930f-4865-a88f-2dd55bd49a33 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.578207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.578518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41eadb-0b0b-4660-a16c-6c6c5093deca service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.579390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.579721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.580095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.585761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.586628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.587083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.587624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] PATH_INFO: `/v3/projects/9249e1a798344c07ac7f828954a3a1bb/users/dd4ed3b4807c47f7a1771e7d01ba3f97/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:15.589895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.590222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9249e1a798344c07ac7f828954a3a1bb, user_id=dd4ed3b4807c47f7a1771e7d01ba3f97, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:15.591637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.594425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:15.610186 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 944/1912] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:15.619545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 969/1913] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:15 2022] PUT /identity/v3/projects/9249e1a798344c07ac7f828954a3a1bb/users/dd4ed3b4807c47f7a1771e7d01ba3f97/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:15.685169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:15.706814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758e56d-0b20-4847-9f45-7e8427c9e1fa service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:15.847674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:15.848595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:15.849247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:15.849622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:16.027345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:16.033357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91261bd9-22a6-443a-9a7e-302beb1a2098 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:16.069581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 945/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:17.028116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:17.163115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bb7f0-3a7e-46ef-a611-33a8a82e5de7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:17.172932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:17.174672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:17.175251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:17.176861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] PATH_INFO: `/v3/users/d8eebdb35d754b39b1b551a25e5195cb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:17.178794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8eebdb35d754b39b1b551a25e5195cb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:17.180796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:17.227786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:17.307815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8eebdb35d754b39b1b551a25e5195cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.309385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8eebdb35d754b39b1b551a25e5195cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.330436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8eebdb35d754b39b1b551a25e5195cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.372006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8eebdb35d754b39b1b551a25e5195cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.374119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8eebdb35d754b39b1b551a25e5195cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.399310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 970/1915] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:17 2022] DELETE /identity/v3/users/d8eebdb35d754b39b1b551a25e5195cb => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:17.404546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:17.405230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:17.405658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:17.437392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:17.440287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:17.453715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:17.454989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:17.493675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:17.549456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653f62-7816-42ef-b8c2-d559af050c47 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:17.553377 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 946/1916] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:17.562069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:17.564028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:17.567087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:17.567748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:17.568025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:17.568314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] PATH_INFO: `/v3/users/2691f882a7c74fe9b29a8675ccb554c4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:17.569656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2691f882a7c74fe9b29a8675ccb554c4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:17.572221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:17.726416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2691f882a7c74fe9b29a8675ccb554c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.727137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2691f882a7c74fe9b29a8675ccb554c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.740593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2691f882a7c74fe9b29a8675ccb554c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.758948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f8153-063b-47e5-9394-65bf1f5b7fde None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:17.765522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2691f882a7c74fe9b29a8675ccb554c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.766333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6777865-2536-4584-96b5-2485e48a1473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2691f882a7c74fe9b29a8675ccb554c4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:17.767328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:17.767957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:17.768232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:17.768501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:17.773478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:17.797244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 971/1917] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:17 2022] DELETE /identity/v3/users/2691f882a7c74fe9b29a8675ccb554c4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:17.802365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:17.803068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:17.803358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:17.808311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:17.809786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:17.834272 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 947/1918] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:17.845890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:17.847407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:17.865952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:18.074690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 972/1919] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:18.104302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:18.106587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:18.122786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8194b-6df9-4c67-84b6-3e62f835b029 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:18.219927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:18.220626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:18.221025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:18.221368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:18.235514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:18.237446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:18.252156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 948/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:18.411087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:18.412572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:18.436872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7d878-49eb-4956-93e5-801bb7128baf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:18.449453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:18.450139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:18.450401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:18.450703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] PATH_INFO: `/v3/projects/82024a37b59b4790b869f387793fd5bc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:18.457737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82024a37b59b4790b869f387793fd5bc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:18.459895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:18.585739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82024a37b59b4790b869f387793fd5bc', 'request_id': 'req-35976d8f-67a9-4716-9f6a-85e81c00b8c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.606131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82024a37b59b4790b869f387793fd5bc', 'request_id': 'req-35976d8f-67a9-4716-9f6a-85e81c00b8c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.617942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82024a37b59b4790b869f387793fd5bc', 'request_id': 'req-35976d8f-67a9-4716-9f6a-85e81c00b8c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.659967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] The token cache is being invalidate because project 82024a37b59b4790b869f387793fd5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:18.660266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.663189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.664966 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 973/1921] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:18 2022] DELETE /identity/v3/projects/82024a37b59b4790b869f387793fd5bc => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:18.669248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:18.686526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ab715-c435-4f8e-bdb0-88d4a35bca8d service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:18.801405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:18.802477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:18.802898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:18.803239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] PATH_INFO: `/v3/users/9397b23c06f14ac396c4ecb5e41231dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:18.804834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9397b23c06f14ac396c4ecb5e41231dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:18.807058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:18.842536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:18.911052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9397b23c06f14ac396c4ecb5e41231dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.912098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9397b23c06f14ac396c4ecb5e41231dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.939771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9397b23c06f14ac396c4ecb5e41231dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.971049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9397b23c06f14ac396c4ecb5e41231dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:18.974769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9397b23c06f14ac396c4ecb5e41231dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.051136 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 949/1922] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:18 2022] DELETE /identity/v3/users/9397b23c06f14ac396c4ecb5e41231dc => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:19.278119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:19.284545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:19.310752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35976d8f-67a9-4716-9f6a-85e81c00b8c9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:19.482336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:19.489565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:19.489859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:19.490128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] PATH_INFO: `/v3/projects/0a84d3ddc23746749c1395d42e3d987d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:19.496421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a84d3ddc23746749c1395d42e3d987d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:19.510873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:19.633950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:19.659651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a84d3ddc23746749c1395d42e3d987d', 'request_id': 'req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.672404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a84d3ddc23746749c1395d42e3d987d', 'request_id': 'req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.689270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a84d3ddc23746749c1395d42e3d987d', 'request_id': 'req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.714673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] The token cache is being invalidate because project 0a84d3ddc23746749c1395d42e3d987d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:19.715112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.716401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.718197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 974/1923] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:19 2022] DELETE /identity/v3/projects/0a84d3ddc23746749c1395d42e3d987d => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:19.722243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:19.723771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:19.737775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b010534-87db-4fc7-8e9b-d146ae47aa68 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:19.818933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:19.819864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:19.820146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:19.820437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] PATH_INFO: `/v3/users/24aee5bd1fff4471b4c537d76edd9a7e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:19.823015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=24aee5bd1fff4471b4c537d76edd9a7e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:19.825089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:19.902427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24aee5bd1fff4471b4c537d76edd9a7e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.903252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24aee5bd1fff4471b4c537d76edd9a7e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.927132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24aee5bd1fff4471b4c537d76edd9a7e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.940529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24aee5bd1fff4471b4c537d76edd9a7e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.941403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-383123b8-fb07-485e-8aab-90a7cd6903ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24aee5bd1fff4471b4c537d76edd9a7e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:19.985281 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 950/1924] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:19 2022] DELETE /identity/v3/users/24aee5bd1fff4471b4c537d76edd9a7e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:20.528407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:20.530129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:20.553493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf50cf-87e5-42a9-8566-2ff5d0cf1d0e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:20.651726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:20.652388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:20.652703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:20.652924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] PATH_INFO: `/v3/projects/5af66b8e090845c58ea3e68fe1bd844a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:20.654401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af66b8e090845c58ea3e68fe1bd844a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:20.657912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:20.711799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af66b8e090845c58ea3e68fe1bd844a', 'request_id': 'req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:20.718595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af66b8e090845c58ea3e68fe1bd844a', 'request_id': 'req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:20.725388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af66b8e090845c58ea3e68fe1bd844a', 'request_id': 'req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:20.742329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] The token cache is being invalidate because project 5af66b8e090845c58ea3e68fe1bd844a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:20.743907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:20.745134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6b1b572b-c17e-4084-8ff6-f9a9b0ac4a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:20.747249 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 975/1925] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:20 2022] DELETE /identity/v3/projects/5af66b8e090845c58ea3e68fe1bd844a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:20.752692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:20.753276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:20.753494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:20.778439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:20.781396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:20.807105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:20.875216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 951/1926] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:20.883107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:20.885430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:20.885776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:20.921118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:20.923566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:20.940952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 976/1927] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:20.942175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:20.956232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0245b41-49ad-4ed6-93a7-938b4eed51af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:20.963862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:20.965138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:20.965577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:20.965860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:20.966537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:20.967234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:20.969231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:20.987946 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 952/1928] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:50:20.992374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.012048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d091bf-640c-4bfa-ad19-925581a845ca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.012048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.012048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.012048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.012466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.012466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.016773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.054725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 977/1929] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:20 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:21.056465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.069988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7c09-7dbc-4e5b-8309-e8fbdfe3f9d8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.152518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.153216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.153569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.154208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.155040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.158001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.206831 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 953/1930] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:21.210669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.224819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff6d84-05de-4fc5-b8bf-57f464b0b200 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.235436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.236562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.237035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.237542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.238611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.240514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.255394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 978/1931] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:21.259417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.278452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4e3bd-7806-4072-b285-4b2890d1f6b1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.285639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.286567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.287265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.287768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.289315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.293152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.317010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 954/1932] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:21.320887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.343356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f85f8-24a7-4b47-9de3-e3860b1d6afe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.348646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.350019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.350019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.350342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] PATH_INFO: `/v3/projects/a834d0cdd7f74375b696cfd64005ba3f/users/937906316d794a34bfcf42139e5e836d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.366976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.370033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:21.376067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a834d0cdd7f74375b696cfd64005ba3f, user_id=937906316d794a34bfcf42139e5e836d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.378796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159832a-831a-4cfe-8804-da0ecc06955d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.381569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:21.401790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d9aa-8c0b-4da2-bae0-d43ff6452f58 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.414566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 979/1933] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:21 2022] PUT /identity/v3/projects/a834d0cdd7f74375b696cfd64005ba3f/users/937906316d794a34bfcf42139e5e836d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:21.418480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.418913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.419238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.459529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] MFA Rules not processed for user `937906316d794a34bfcf42139e5e836d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:21.460991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:21.533438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.534411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.534698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.535058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] PATH_INFO: `/v3/projects/38a02cd040484a8c98525fd9cc37d266` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.536620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a02cd040484a8c98525fd9cc37d266)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.538803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.589060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:21.615413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a02cd040484a8c98525fd9cc37d266', 'request_id': 'req-997ac37a-41db-420d-9eac-c3c441598be2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:21.621983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a02cd040484a8c98525fd9cc37d266', 'request_id': 'req-997ac37a-41db-420d-9eac-c3c441598be2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:21.635905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a02cd040484a8c98525fd9cc37d266', 'request_id': 'req-997ac37a-41db-420d-9eac-c3c441598be2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:21.638801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 980/1934] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:21.639923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.694736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fead1-18c3-4882-aec7-7161b306b95d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.698097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] The token cache is being invalidate because project 38a02cd040484a8c98525fd9cc37d266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:21.698897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:21.699545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-997ac37a-41db-420d-9eac-c3c441598be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:21.704500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 955/1935] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:21 2022] DELETE /identity/v3/projects/38a02cd040484a8c98525fd9cc37d266 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:21.709785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c674-f411-47c8-8447-59d7eea4bae4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.710129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c674-f411-47c8-8447-59d7eea4bae4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.710392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c674-f411-47c8-8447-59d7eea4bae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.741549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8ee0c674-f411-47c8-8447-59d7eea4bae4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:21.742832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8ee0c674-f411-47c8-8447-59d7eea4bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:21.783266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.784317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.784806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.785429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.786309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.788696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.824634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 981/1936] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:21.825849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.827383 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 956/1937] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:21.834834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:21.836454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.836929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.837399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.864301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:21.866058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:21.905481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246105-f335-491b-9ab4-792927c40a3f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.910558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.911184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.911459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.911728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.912357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.914303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.931944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:21.939095 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 982/1938] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:21.944406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.949607 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 957/1939] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:21.954844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:21.959272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93846f-03e5-4b16-b751-45ed659fd836 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.965457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.966072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.966342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.966666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.968307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.970046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.974887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53b571-56b1-4da5-9d55-57a58e839426 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:21.984115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:21.987564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:21.987769 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 983/1940] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:21.987865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:21.987865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:21.987865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:21.987865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:21.988494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:21.991444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.008356 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 958/1941] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:22.008560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7589d-f6e7-4f60-9fb6-459c500d297f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.009797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.011649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.012358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.012762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.013137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] PATH_INFO: `/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57/users/6d8f8464e20643fe8f124c492a3e77af/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.023958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3dfe6-a226-498d-a861-e4d4739ab3f4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.029935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.031352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.032165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.032449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.033364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.036122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.039010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac98c9a3687443f986ab9f6fcc2fd57, user_id=6d8f8464e20643fe8f124c492a3e77af, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.043133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.072592 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 984/1942] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:21 2022] PUT /identity/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57/users/6d8f8464e20643fe8f124c492a3e77af/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:22.078159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.112206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 959/1943] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:22.116809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.172541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:22.203814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f69fc-d04a-4bcc-a0f4-f6817200e2d4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.253207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6645d-0f06-4aa2-af2a-49f47a555f24 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.260748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.262029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.263263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.263790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.266392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.270914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.309097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.310369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.311275 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 960/1944] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:22.311730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.312173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.313001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.314996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.315709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.333512 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 985/1945] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:22.335395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.340197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039dd8-20c8-4ffa-8994-183289753ca6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.354159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ec640-6b79-4f69-90e2-99f894101e36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.355551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.356576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.356859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.357134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.358137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.359920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.362306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.363213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.364006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.364006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] PATH_INFO: `/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57/users/6d8f8464e20643fe8f124c492a3e77af/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.365958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac98c9a3687443f986ab9f6fcc2fd57, user_id=6d8f8464e20643fe8f124c492a3e77af, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.369849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abd7ca-7418-4c8b-976c-b330ef5be4be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.390553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 986/1946] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:22 2022] PUT /identity/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57/users/6d8f8464e20643fe8f124c492a3e77af/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:22.392403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 961/1947] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:22.396966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd707e3-77ff-4b57-9030-4e30d77b37cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.397468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd707e3-77ff-4b57-9030-4e30d77b37cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.399157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd707e3-77ff-4b57-9030-4e30d77b37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.400168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.420407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab6561-af60-4698-bef2-bde9ba5c6be9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.470384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5cd707e3-77ff-4b57-9030-4e30d77b37cb None None] MFA Rules not processed for user `6d8f8464e20643fe8f124c492a3e77af`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:22.487178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5cd707e3-77ff-4b57-9030-4e30d77b37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:22.566009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.567185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.567834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.568336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.569176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.571864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.593763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 962/1948] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:22.598293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.612991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12e5bb-2672-45c8-8d02-4f070b04aa1a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.620866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.621858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.622729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.623215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] PATH_INFO: `/v3/projects/a2b7e30f79bf42e497c53c091ad0a106/users/9c03c67e0c4c4de7a186f28ac10d0b92/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.650897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:22.680581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b7e30f79bf42e497c53c091ad0a106, user_id=9c03c67e0c4c4de7a186f28ac10d0b92, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.683393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.703245 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 963/1949] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:22 2022] PUT /identity/v3/projects/a2b7e30f79bf42e497c53c091ad0a106/users/9c03c67e0c4c4de7a186f28ac10d0b92/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:22.704013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 987/1950] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:22.704536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.709156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016ed54-ff50-4153-8e0e-07bc6fba7748 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.709623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016ed54-ff50-4153-8e0e-07bc6fba7748 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.710041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016ed54-ff50-4153-8e0e-07bc6fba7748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.732648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1016ed54-ff50-4153-8e0e-07bc6fba7748 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:22.734741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1016ed54-ff50-4153-8e0e-07bc6fba7748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:22.821415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379710ae-46ee-49fc-a16b-62bc68dfeec8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.828280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.828972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.829300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.829819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.830926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:22.831096 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 988/1951] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:22.831187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.832583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff884-e988-4791-b847-6cf67b6bde44 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.832868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.833154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff884-e988-4791-b847-6cf67b6bde44 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.833568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaff884-e988-4791-b847-6cf67b6bde44 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.842567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 989/1952] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:22.844191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.844480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.844753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.848120 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 964/1953] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:22.849093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.851471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:22.864339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:22.875310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:22.876530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:22.922463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846914cb-4f1f-4ad0-b95a-98b90207bc28 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.926186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.926846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.927126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.927399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.938701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.940072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.950541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 965/1954] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:22.951350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.955337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 990/1955] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:22.970026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:22.971887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:22.975757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80be16b-acff-4fb4-ad0a-8f6b42ed57f4 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:22.981321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:22.982082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:22.983129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:22.983532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:22.984236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:22.986239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:22.991674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d0124-c527-4093-a7ad-8ecd579bbe1d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:23.000357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:23.001889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:23.002404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:23.003014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:23.016303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:23.026357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:23.038406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 966/1956] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:22 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:23.042406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:23.070969 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 991/1957] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:23.166399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f5626-8116-42e2-b3e0-f8cb9f507aa0 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:23.174586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:23.175458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:23.175931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:23.176313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:23.177071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:23.177488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:23.187546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beddd15-0a8e-4e9c-8186-89e9151ae5b8 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:23.205659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 967/1958] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:23.219859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:23.234792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731db240-342d-491e-98a2-829fc2178d65 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:23.301295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:23.333541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:23.335934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:23.336360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:23.336740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:23.359415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:23.361792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:23.407338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 992/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:23.447103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484baa5-2834-422f-9188-fe47d1dcfba5 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:23.447802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484baa5-2834-422f-9188-fe47d1dcfba5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:23.448297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484baa5-2834-422f-9188-fe47d1dcfba5 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:23.451466 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 968/1960] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:23.457163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:23.470225 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 24 20:50:23.470698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:23.481122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7899f9-282c-4f18-92b3-ef787e876795 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:23.486788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:23.487698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:23.488420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:23.488420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects/bea627807de3458c9376652ac768daa8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:23.490485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:get_project(project_id=bea627807de3458c9376652ac768daa8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:23.492705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:23.508896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 993/1961] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:23 2022] GET /identity/v3/projects/bea627807de3458c9376652ac768daa8 => generated 369 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:24.679602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c78a-d810-4edd-b055-fa9fd4e95c7c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:24.680174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c78a-d810-4edd-b055-fa9fd4e95c7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:24.680174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5c78a-d810-4edd-b055-fa9fd4e95c7c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:24.682111 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 969/1962] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:24.688163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:24.690127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:24.721791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:24.745004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b363e-d526-4c7b-a8be-bbb56948b80c tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:24.757036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:24.757697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:24.757971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:24.758249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects/bea627807de3458c9376652ac768daa8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:24.759434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:get_project(project_id=bea627807de3458c9376652ac768daa8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:24.762555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:24.765799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 994/1963] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:24 2022] GET /identity/v3/projects/bea627807de3458c9376652ac768daa8 => generated 369 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:24.853460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5c78a-d810-4edd-b055-fa9fd4e95c7c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:24.887959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5c78a-d810-4edd-b055-fa9fd4e95c7c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:24.901031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:24.901819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:24.902204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:24.902974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:24.903967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:24.905083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:24.907578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:24.938160 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 970/1964] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:24.939473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:24.958732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b603c-26ae-4f7d-a3d1-8bcd9c13825d tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:24.973174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:24.973806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:24.974105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:24.974422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:24.975105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:24.980141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdde07-0a10-41d7-8f16-b64b93d80912 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:25.036228 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 995/1965] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:24 2022] POST /identity/v3/users => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:25.037252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:25.057897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea85928-b846-42fb-9476-49df4b87052a tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:25.073220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:25.073976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:25.074153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:25.074389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:25.075106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:25.075409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:25.077593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:25.098818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 971/1966] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:25.595176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a89-d288-4cd9-84e4-799549e903c2 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:25.595713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a89-d288-4cd9-84e4-799549e903c2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:25.596154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc1a89-d288-4cd9-84e4-799549e903c2 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:25.597967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 996/1967] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:25.614989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:25.641757 ncdv-master-1 devstack@keystone.service[99133]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. Sep 24 20:50:25.642071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:25.668030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437695-fd34-41bc-9a5d-1e2cef1f3f3f tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:25.684716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:25.685429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:25.685713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:25.686084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects/bea627807de3458c9376652ac768daa8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:25.687633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:get_project(project_id=bea627807de3458c9376652ac768daa8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:25.695069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:25.699307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 972/1968] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:25 2022] GET /identity/v3/projects/bea627807de3458c9376652ac768daa8 => generated 369 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:50:26.942402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5917f1-34bb-46c1-a9db-835d4b19a33a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:26.942961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5917f1-34bb-46c1-a9db-835d4b19a33a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:26.943052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5917f1-34bb-46c1-a9db-835d4b19a33a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:26.950190 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 997/1969] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:26.956680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:26.991969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:27.022855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d1ead-8923-4f0e-99c1-79c422f35341 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:27.030830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:27.031868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:27.032353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:27.032852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects/bea627807de3458c9376652ac768daa8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:27.034384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:get_project(project_id=bea627807de3458c9376652ac768daa8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:27.037127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:27.053652 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 973/1970] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:26 2022] GET /identity/v3/projects/bea627807de3458c9376652ac768daa8 => generated 369 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:27.076747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5917f1-34bb-46c1-a9db-835d4b19a33a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:27.110179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5917f1-34bb-46c1-a9db-835d4b19a33a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:27.130982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:27.132844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:27.133132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:27.133409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/users/9929bcabada74355bb90f3b31590c07b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:27.154653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:delete_user(user_id=9929bcabada74355bb90f3b31590c07b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:27.156834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:27.217557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9929bcabada74355bb90f3b31590c07b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.219042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9929bcabada74355bb90f3b31590c07b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.236593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9929bcabada74355bb90f3b31590c07b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.255559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9929bcabada74355bb90f3b31590c07b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.256731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9929bcabada74355bb90f3b31590c07b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.274942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 998/1971] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:27 2022] DELETE /identity/v3/users/9929bcabada74355bb90f3b31590c07b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:27.280083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:27.282535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:27.297793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e0ce2-a08d-4610-934a-b92a5e39f724 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:27.350975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:27.394019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8f8464e20643fe8f124c492a3e77af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ac98c9a3687443f986ab9f6fcc2fd57', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:27.396546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:27.396546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:27.396719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] PATH_INFO: `/v3/projects/bea627807de3458c9376652ac768daa8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:27.398055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bea627807de3458c9376652ac768daa8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:27.401142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:27.538519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea627807de3458c9376652ac768daa8', 'request_id': 'req-9f24addb-5643-47c5-990a-1fcdd70d2e3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.564991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea627807de3458c9376652ac768daa8', 'request_id': 'req-9f24addb-5643-47c5-990a-1fcdd70d2e3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.579681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea627807de3458c9376652ac768daa8', 'request_id': 'req-9f24addb-5643-47c5-990a-1fcdd70d2e3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.600578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] The token cache is being invalidate because project bea627807de3458c9376652ac768daa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:27.600970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.601719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f24addb-5643-47c5-990a-1fcdd70d2e3e tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.610095 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 974/1972] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:27 2022] DELETE /identity/v3/projects/bea627807de3458c9376652ac768daa8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:27.610621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:27.612905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:27.735593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b96581-d81e-4d4b-ade9-dd141c5d7016 tempest-QuotasAdminTestV257-361241291 tempest-QuotasAdminTestV257-361241291-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:27.753434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:27.839284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:27.840324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:27.840909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:27.841599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] PATH_INFO: `/v3/users/937906316d794a34bfcf42139e5e836d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:27.843360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937906316d794a34bfcf42139e5e836d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:27.845541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:27.934166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937906316d794a34bfcf42139e5e836d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.934978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937906316d794a34bfcf42139e5e836d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.946946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937906316d794a34bfcf42139e5e836d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.967454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937906316d794a34bfcf42139e5e836d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.970123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937906316d794a34bfcf42139e5e836d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:27.993543 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 999/1973] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:27 2022] DELETE /identity/v3/users/937906316d794a34bfcf42139e5e836d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:27.999323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:27.999929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:27.999929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:28.030264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:28.031865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:28.162196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 975/1974] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:28.193170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:28.195413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:28.313991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26e5c8-25e1-4116-b61e-e51ff756c955 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:28.324561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:28.328404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:28.330125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:28.330412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:28.361238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:28.363032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:28.397530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1000/1975] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:29.271805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:29.273833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:29.289419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e632aaa-5391-4414-86c5-46acf75ef516 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:29.303273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:29.304368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:29.305015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:29.305993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] PATH_INFO: `/v3/projects/a834d0cdd7f74375b696cfd64005ba3f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:29.312268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a834d0cdd7f74375b696cfd64005ba3f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:29.314707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:29.428428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a834d0cdd7f74375b696cfd64005ba3f', 'request_id': 'req-9dc93baf-682b-4410-b979-914c303b71fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.441559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a834d0cdd7f74375b696cfd64005ba3f', 'request_id': 'req-9dc93baf-682b-4410-b979-914c303b71fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.454183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a834d0cdd7f74375b696cfd64005ba3f', 'request_id': 'req-9dc93baf-682b-4410-b979-914c303b71fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.455743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:29.483210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] The token cache is being invalidate because project a834d0cdd7f74375b696cfd64005ba3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:29.483491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.484249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9dc93baf-682b-4410-b979-914c303b71fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.489029 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 976/1976] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:29 2022] DELETE /identity/v3/projects/a834d0cdd7f74375b696cfd64005ba3f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:29.490055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:29.503084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ca0ca-64ba-47f5-8ba6-5d195a799eb7 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:29.595049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:29.640434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:29.641316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:29.641685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:29.642059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] PATH_INFO: `/v3/users/6d8f8464e20643fe8f124c492a3e77af` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:29.644961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8f8464e20643fe8f124c492a3e77af)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:29.650615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:29.722472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8f8464e20643fe8f124c492a3e77af'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.723338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8f8464e20643fe8f124c492a3e77af'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.735524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8f8464e20643fe8f124c492a3e77af'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.759178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8f8464e20643fe8f124c492a3e77af'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.761248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8f8464e20643fe8f124c492a3e77af'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:29.785973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1001/1977] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:29 2022] DELETE /identity/v3/users/6d8f8464e20643fe8f124c492a3e77af => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:30.426211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:30.426723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:30.426815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:30.487033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] MFA Rules not processed for user `9c03c67e0c4c4de7a186f28ac10d0b92`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:30.488930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:30.640003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:30.640811 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 977/1978] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:30.641928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:30.755673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a1a1b-8f3c-42be-aa98-875c1dbe1579 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:30.761564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:30.762231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:30.762739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:30.763214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:30.764552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:30.766678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:30.810216 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1002/1979] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:30 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:30.811236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:30.826360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64003f0a-5e6d-4893-b411-45f41134a8db None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:30.910078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:30.912008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:30.912503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:30.913043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:30.914182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:30.917209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:30.943274 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 978/1980] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:30 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:30.947125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:30.963192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af4a1-40d2-43b2-b720-b0479da34e3f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:30.970462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:30.971266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:30.971620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:30.972002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:30.972726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:30.974853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.012425 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1003/1981] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:31.014309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:31.032695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13506d6-4aad-4271-ab06-1d574c5b1cdc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:31.045576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:31.048689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.049129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.049415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] PATH_INFO: `/v3/projects/5d5c7562e0a14c8997a94b916a0871de/users/68910ac4980a4f74895c0a5ab25199c9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.102873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c7562e0a14c8997a94b916a0871de, user_id=68910ac4980a4f74895c0a5ab25199c9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:31.105645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.126161 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 979/1982] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:31 2022] PUT /identity/v3/projects/5d5c7562e0a14c8997a94b916a0871de/users/68910ac4980a4f74895c0a5ab25199c9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:31.130245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:31.156285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2506474-f2e7-442b-9bfc-b29f67f6634d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:31.200153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:31.239069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:31.239758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.240039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.240307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.241193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:31.242915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.264213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1004/1983] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:31.265221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:31.265335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:31.268540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:31.282447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a227db0-5843-4811-80b6-d67aefdbb106 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:31.288962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:31.289880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.290351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.290832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] PATH_INFO: `/v3/projects/5d5c7562e0a14c8997a94b916a0871de/users/68910ac4980a4f74895c0a5ab25199c9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.296413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c7562e0a14c8997a94b916a0871de, user_id=68910ac4980a4f74895c0a5ab25199c9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:31.299159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.305084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c22059-f0a0-49c6-900e-d1df2c784cfc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:31.317826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:31.318350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.318474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.318761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] PATH_INFO: `/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.319914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac98c9a3687443f986ab9f6fcc2fd57)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:31.320303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1005/1984] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:31 2022] PUT /identity/v3/projects/5d5c7562e0a14c8997a94b916a0871de/users/68910ac4980a4f74895c0a5ab25199c9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:31.322647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.342394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:31.345607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:31.451246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:31.457845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac98c9a3687443f986ab9f6fcc2fd57', 'request_id': 'req-4424edd6-65e1-42aa-b978-1804a7a03b54'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:31.482647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac98c9a3687443f986ab9f6fcc2fd57', 'request_id': 'req-4424edd6-65e1-42aa-b978-1804a7a03b54'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:31.496250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac98c9a3687443f986ab9f6fcc2fd57', 'request_id': 'req-4424edd6-65e1-42aa-b978-1804a7a03b54'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:31.521748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] The token cache is being invalidate because project 8ac98c9a3687443f986ab9f6fcc2fd57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:31.522045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:31.524931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4424edd6-65e1-42aa-b978-1804a7a03b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:31.528297 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 980/1985] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:31 2022] DELETE /identity/v3/projects/8ac98c9a3687443f986ab9f6fcc2fd57 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:31.532378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947453-5363-42fb-8258-9c804d61132e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.532888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947453-5363-42fb-8258-9c804d61132e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.534238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c947453-5363-42fb-8258-9c804d61132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.541021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74016-da6d-4b9f-b9b5-e6820ecfea59 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:31.576600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3c947453-5363-42fb-8258-9c804d61132e None None] MFA Rules not processed for user `dd4ed3b4807c47f7a1771e7d01ba3f97`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:31.578947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c947453-5363-42fb-8258-9c804d61132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:31.693621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:31.695067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.698853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.699459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.703759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:31.715371 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 981/1986] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:31.717218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.718913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.720341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:31.764569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:31.767685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:31.821952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:31.854549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:31.861749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6525f-5a35-40ca-a67e-2e4a8bcb662a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:31.897658 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1006/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:31.956168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 982/1988] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:31.963771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:31.964331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:31.964780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.028589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:32.030572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:32.062796 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1007/1989] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:32.065739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.080686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1dd9-d532-4bcb-b79d-56cce88badcf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.095328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.096055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.099248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.099626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.100390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:32.100772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.106420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.143207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 983/1990] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:32.144332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.159639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df2d87-517b-458f-b108-6423b41547d4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.167988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.168620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.168897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.170442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.171132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.173765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.224994 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1008/1991] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:32 2022] POST /identity/v3/projects => generated 391 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:32.226262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.246067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba265f-d312-4347-bd15-93658a24a399 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.372361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:32.409625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.410250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.410594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.410862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.411502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.413444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.455279 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 984/1992] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:32 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:32.458967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.476005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d080df9-7920-4ef8-a6bf-794e6e80b859 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.492454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.493022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.493405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.493774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.494561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.496275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.517249 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1009/1993] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:32.521237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.539405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f489-ec99-4100-9c54-6142a87d0dc9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.544933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.545890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.546321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.546768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.547701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.549487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480ebb2-6b5a-440a-8fa7-67980072b883 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.565774 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 985/1994] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:32.566601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:32.578974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc499-59c9-428c-a980-3f853b4fe1d2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:32.593546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:32.596915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.597553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.597989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] PATH_INFO: `/v3/projects/899574bf215744b29695dc6128d4c8dd/users/8df2631dab974889a95bed3f863c0219/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.636329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899574bf215744b29695dc6128d4c8dd, user_id=8df2631dab974889a95bed3f863c0219, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:32.642802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:32.674557 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1010/1995] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:32 2022] PUT /identity/v3/projects/899574bf215744b29695dc6128d4c8dd/users/8df2631dab974889a95bed3f863c0219/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:32.685065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:32.685487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:32.685932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:32.747569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] MFA Rules not processed for user `8df2631dab974889a95bed3f863c0219`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:32.749279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:32.891399 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 986/1996] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:32.895479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.047033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4001620-4fa3-4023-85ad-99e0002407c0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.062533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.063229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.063513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.063780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.064975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.067464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.115496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1011/1997] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:32 2022] POST /identity/v3/projects => generated 393 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:33.119293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.137530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fc4bd-eddd-4d28-a8db-22dcbab7a6af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.265091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.265733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.266018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.266307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.267239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.270054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.311898 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 987/1998] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:33 2022] POST /identity/v3/users => generated 303 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:33.315525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.331198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8b86b-320c-484f-827d-5b2af739f6b0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.338536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.339118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.339423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.339663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.340369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.342577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.358977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1012/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:33.362430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.387684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f2563-2a07-46aa-bff6-f4525e5f6c03 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.404188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.405185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.405470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.405896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] PATH_INFO: `/v3/projects/93c5cd84b7c44b258f25643417650f60/users/d781afdc483d42c1bdcfbaa1294a4b31/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.437148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c5cd84b7c44b258f25643417650f60, user_id=d781afdc483d42c1bdcfbaa1294a4b31, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.442235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.462611 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 988/2000] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:33 2022] PUT /identity/v3/projects/93c5cd84b7c44b258f25643417650f60/users/d781afdc483d42c1bdcfbaa1294a4b31/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:33.466723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.480028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49afaf58-e750-4a5e-b7ca-6624680ad571 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.582966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:33.594088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.595167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.595595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.596055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.596863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.599401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203a455-e374-4b3d-9da3-9517a59c1847 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.614848 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1013/2001] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:33.615663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:33.627719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3268a81-70bc-4c96-ac2e-dd42b51c6ec2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:33.640910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:33.641849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.642322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.642856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] PATH_INFO: `/v3/projects/93c5cd84b7c44b258f25643417650f60/users/d781afdc483d42c1bdcfbaa1294a4b31/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.646168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c5cd84b7c44b258f25643417650f60, user_id=d781afdc483d42c1bdcfbaa1294a4b31, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:33.649410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e97e6-914a-4e6a-ad8f-0a2d0406e5d5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:33.667357 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 989/2002] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:33 2022] PUT /identity/v3/projects/93c5cd84b7c44b258f25643417650f60/users/d781afdc483d42c1bdcfbaa1294a4b31/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:33.678158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.678749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.679175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.726344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] MFA Rules not processed for user `d781afdc483d42c1bdcfbaa1294a4b31`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:33.728452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:33.888996 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1014/2003] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:33.900540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cded-982b-444d-a7dd-271782b2d140 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:33.900831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cded-982b-444d-a7dd-271782b2d140 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:33.901200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17cded-982b-444d-a7dd-271782b2d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:33.916565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:33.956038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8e17cded-982b-444d-a7dd-271782b2d140 None None] MFA Rules not processed for user `8df2631dab974889a95bed3f863c0219`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:33.957293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e17cded-982b-444d-a7dd-271782b2d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:34.040506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 990/2004] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:34.429036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:34.430861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:34.643512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ff806-c370-42fd-b67f-2940642a5bfb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:34.663937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:34.664577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:34.664861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:34.665138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:34.699144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:34.702281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:34.720568 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1015/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:38.050740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d3c0-f18d-4941-84cc-4e65ebb96c7c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:38.051586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d3c0-f18d-4941-84cc-4e65ebb96c7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:38.052207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4d3c0-f18d-4941-84cc-4e65ebb96c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:38.101905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8cf4d3c0-f18d-4941-84cc-4e65ebb96c7c None None] MFA Rules not processed for user `50d8a91794144d73a8942f5c3bac8cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:38.106164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8cf4d3c0-f18d-4941-84cc-4e65ebb96c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:38.263416 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 991/2006] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:40.926230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:40.928091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:40.962193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2ab2-8175-4690-992c-2a4664bb11ab service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:40.976891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:40.977521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:40.977800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:40.978072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:40.993101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:40.996648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e1c89-cf86-4454-947a-b4b0eae9cc88 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:41.015151 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1016/2007] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:42.881260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:42.881831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:42.881831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:42.883198 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 992/2008] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:42.897498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11435-636a-42e7-94e5-10cb9662b63c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:42.898382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11435-636a-42e7-94e5-10cb9662b63c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:42.899041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d11435-636a-42e7-94e5-10cb9662b63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:42.934960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-92d11435-636a-42e7-94e5-10cb9662b63c None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:42.937691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92d11435-636a-42e7-94e5-10cb9662b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:43.022227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:43.062091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1017/2009] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:43.084726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:43.086524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:43.133068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8b40-4c01-47c5-a10b-7662b20060ee None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:43.147752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:43.149197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:43.149859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:43.152101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:43.177690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:43.187642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:43.205262 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 993/2010] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:45.704903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07ee37-9c28-4bd7-8b18-8653456d1038 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:45.705724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07ee37-9c28-4bd7-8b18-8653456d1038 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:45.706275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07ee37-9c28-4bd7-8b18-8653456d1038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:45.754648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-de07ee37-9c28-4bd7-8b18-8653456d1038 None None] MFA Rules not processed for user `68910ac4980a4f74895c0a5ab25199c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:45.756348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de07ee37-9c28-4bd7-8b18-8653456d1038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:45.854747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:45.856919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:45.865305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1018/2011] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:45.876624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:45.876914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:45.877193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:45.915998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] MFA Rules not processed for user `9c03c67e0c4c4de7a186f28ac10d0b92`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:45.917485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.008277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad4a98-169f-4d0f-9b52-e68c8ee3a8bf service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:46.021534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:46.023208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:46.023985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:46.027086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] PATH_INFO: `/v3/users/7af773989be84d47a75f107f9acc8c69` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:46.030018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af773989be84d47a75f107f9acc8c69)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:46.036707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:46.045808 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1019/2012] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:46.065088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:46.093551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09f4fb-f765-4929-8b86-b12a95c94869 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:46.108634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:46.110365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:46.111262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:46.111895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:46.114759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.132270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:46.135539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fa16-ebeb-49d4-9350-7c3ea09b1556 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:46.141201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:46.156171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af773989be84d47a75f107f9acc8c69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:46.157488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af773989be84d47a75f107f9acc8c69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:46.161537 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1020/2013] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:46.181594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af773989be84d47a75f107f9acc8c69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:46.201719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af773989be84d47a75f107f9acc8c69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:46.202876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af773989be84d47a75f107f9acc8c69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:46.248004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 994/2014] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:45 2022] DELETE /identity/v3/users/7af773989be84d47a75f107f9acc8c69 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:46.262794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:46.263139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:46.263426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:46.264900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:46.266615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.304007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:46.306328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.412360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fac1b-27bc-454d-966f-bdbde2c337dd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:46.427408 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1021/2015] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:46.438663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:46.441317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:46.442240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:46.443313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:46.447795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.459460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:46.461583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:46.470077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:46.600651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208588c9-efd4-492c-8597-27f99d812fe9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:46.604894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:46.606671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:46.606671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:46.606932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:46.611088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:46.617547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:46.618407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:46.618939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:46.632313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1022/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:46.654085 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 995/2017] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:47.576592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:47.578673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:47.603317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6339a-8438-4a9c-97c5-fad51408b336 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:47.617674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:47.618615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:47.619076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:47.619518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] PATH_INFO: `/v3/projects/6ea3cc3ecfc941758415b8324faf7644` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:47.620889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea3cc3ecfc941758415b8324faf7644)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:47.624377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:47.738742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea3cc3ecfc941758415b8324faf7644', 'request_id': 'req-e66cfb64-9783-443c-9f24-5344f9558edb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:47.759301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea3cc3ecfc941758415b8324faf7644', 'request_id': 'req-e66cfb64-9783-443c-9f24-5344f9558edb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:47.805012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea3cc3ecfc941758415b8324faf7644', 'request_id': 'req-e66cfb64-9783-443c-9f24-5344f9558edb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:47.836138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] The token cache is being invalidate because project 6ea3cc3ecfc941758415b8324faf7644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:47.836441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:47.837115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:47.843272 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1023/2018] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:47 2022] DELETE /identity/v3/projects/6ea3cc3ecfc941758415b8324faf7644 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:47.844868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:47.876365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8b002-9c02-4d9b-bbe3-54bd225c040d service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:48.032019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:48.064121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:48.065944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:48.069843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:48.070722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.071077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.071481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] PATH_INFO: `/v3/users/64a47c4d366e4af88ff0386e79158e85` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.073539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a47c4d366e4af88ff0386e79158e85)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:48.077185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:48.092831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cfb64-9783-443c-9f24-5344f9558edb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:48.107453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:48.107964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.110731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.112021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] PATH_INFO: `/v3/users/2c01bfb1e8464616aceb3b73c958eb3f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.115790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c01bfb1e8464616aceb3b73c958eb3f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:48.124992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:48.154284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a47c4d366e4af88ff0386e79158e85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.156670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a47c4d366e4af88ff0386e79158e85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.167364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a47c4d366e4af88ff0386e79158e85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.201672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a47c4d366e4af88ff0386e79158e85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.204014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e489ac6f-9936-4127-a161-16ebd76834af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a47c4d366e4af88ff0386e79158e85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.257969 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 996/2019] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:47 2022] DELETE /identity/v3/users/64a47c4d366e4af88ff0386e79158e85 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:48.267589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c01bfb1e8464616aceb3b73c958eb3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.268308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c01bfb1e8464616aceb3b73c958eb3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.283435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c01bfb1e8464616aceb3b73c958eb3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.331653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c01bfb1e8464616aceb3b73c958eb3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.332545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37f575b7-e19f-4036-b77c-fa95e4187d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c01bfb1e8464616aceb3b73c958eb3f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:48.353406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:48.395471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1024/2020] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:48 2022] DELETE /identity/v3/users/2c01bfb1e8464616aceb3b73c958eb3f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:48.402980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.403303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.405090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.476413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:48.487198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:48.623912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42beb56-982b-4dba-b6d1-d5827e8d27b1 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.624468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42beb56-982b-4dba-b6d1-d5827e8d27b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.624468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42beb56-982b-4dba-b6d1-d5827e8d27b1 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.626037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1025/2021] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:48.645501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.646219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.646756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.680254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 997/2022] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:48.687906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:48.692284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:48.786282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:48.787762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:48.807685 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1026/2023] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:48.909935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:48.922343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1b542-1263-40f6-afa3-427e768c5513 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:48.933469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24260a0-7766-40a7-885e-66115d2d2c7c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:48.935653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:48.936902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.937187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.937463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.944132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:48.944765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:48.946861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:48.947629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:48.968815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:48.973867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:48.988655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:48.990599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:49.005172 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1027/2024] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:49.010202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 998/2025] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:49.378300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:49.379894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:49.561262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827a6fe-7159-491b-b1d6-db9bb4981aef service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:49.567214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:49.567835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:49.568169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:49.568451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:49.590891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:49.610960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:49.613881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e284ef3-6e20-4b2a-b167-155505b0eeae service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:49.626760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1028/2026] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:49.705761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:49.707304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:49.732457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb6bd-ff18-42df-8195-484916b7b151 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:49.756555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:49.757209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:49.757482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:49.757757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] PATH_INFO: `/v3/projects/79afe6f2d12f45e994b77b96c44f8239` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:49.760425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79afe6f2d12f45e994b77b96c44f8239)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:49.762889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:49.804658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:49.921951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79afe6f2d12f45e994b77b96c44f8239', 'request_id': 'req-def93cab-176a-4582-a6e4-1c034c1117c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:49.931066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79afe6f2d12f45e994b77b96c44f8239', 'request_id': 'req-def93cab-176a-4582-a6e4-1c034c1117c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:49.945184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79afe6f2d12f45e994b77b96c44f8239', 'request_id': 'req-def93cab-176a-4582-a6e4-1c034c1117c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:49.969440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34acfd9-fc19-4600-9a54-df08e7d18eeb None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:49.969640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] The token cache is being invalidate because project 79afe6f2d12f45e994b77b96c44f8239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:49.969756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34acfd9-fc19-4600-9a54-df08e7d18eeb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:49.969866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:49.969956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34acfd9-fc19-4600-9a54-df08e7d18eeb None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:49.970530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-def93cab-176a-4582-a6e4-1c034c1117c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:49.971957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1029/2027] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:49.981886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 999/2028] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:49 2022] DELETE /identity/v3/projects/79afe6f2d12f45e994b77b96c44f8239 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:49.986766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c038-13fe-4c96-a30b-abf55799b6a2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:49.987049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c038-13fe-4c96-a30b-abf55799b6a2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:49.987326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7c038-13fe-4c96-a30b-abf55799b6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.001917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.002438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.002886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.050239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:50.051716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:50.085831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-37a7c038-13fe-4c96-a30b-abf55799b6a2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:50.087377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-37a7c038-13fe-4c96-a30b-abf55799b6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:50.212301 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1000/2029] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:50.234742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:50.236172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:50.250014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1030/2030] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:50.259219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.259872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.260695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.341212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:50.346752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:50.400251 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1031/2031] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:50.406614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:50.441235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcfc58-3cbb-4184-aafa-37c0d423a07f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:50.450754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd61d-c606-459c-9720-c0aff40cf05b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:50.453186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:50.453737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.454013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.454281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.454983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:50.455285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:50.456892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:50.461685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:50.462310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.463627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.463916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.491560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1032/2032] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:50.501092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:50.504029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:50.505513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:50.537855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5548f6-fab4-457f-9b93-6cc36214cfc5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:50.549225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1001/2033] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:50.559811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:50.561802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.561914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.562195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.562868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:50.564801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:50.582310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:50.624667 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1033/2034] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:50 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:50.650986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:50.671935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42852dcf-10e9-46be-a626-643ead9da7d7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:50.805485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:50.888607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:50.890710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.890995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.891283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.891917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:50.894339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:50.949353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1002/2035] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:50 2022] POST /identity/v3/users => generated 289 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:50.950957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:50.964612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55db59a-d78a-4da1-a97f-fb2d3cdff355 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:50.978478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:50.979607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:50.980232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:50.982241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:50.983532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:50.985420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.009043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:51.011276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:51.020472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1034/2036] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:51.021626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:51.037033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09b289-f40a-43bb-a8c7-8b645e9782e8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:51.040425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4c886-bc1b-42f6-8d4e-69c1a669b085 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:51.049800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:51.050321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:51.051243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.051369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.051563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.052995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] PATH_INFO: `/v3/projects/a9e62e74103b45cc81dfe7900c0baf5a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.056633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e62e74103b45cc81dfe7900c0baf5a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:51.061165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.063299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.063749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.064516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:51.067145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.126107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1035/2037] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:51.130997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:51.157650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb533-79ea-4ada-858a-ac2e8cbd630b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:51.198473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e62e74103b45cc81dfe7900c0baf5a', 'request_id': 'req-1eea2178-59da-4299-9874-e15dbfc818ae'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.211601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e62e74103b45cc81dfe7900c0baf5a', 'request_id': 'req-1eea2178-59da-4299-9874-e15dbfc818ae'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.225377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e62e74103b45cc81dfe7900c0baf5a', 'request_id': 'req-1eea2178-59da-4299-9874-e15dbfc818ae'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.253038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] The token cache is being invalidate because project a9e62e74103b45cc81dfe7900c0baf5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:51.253258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.254741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.260988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1003/2038] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:51 2022] DELETE /identity/v3/projects/a9e62e74103b45cc81dfe7900c0baf5a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:51.267237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:51.292635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:51.294997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.295941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.297225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:51.297611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] PATH_INFO: `/v3/projects/652de1d5c4614bbfaafe5df4628052b9/users/974ae63b8eb24baa9df89f38a5ad63be/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.325236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea2178-59da-4299-9874-e15dbfc818ae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:51.328210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652de1d5c4614bbfaafe5df4628052b9, user_id=974ae63b8eb24baa9df89f38a5ad63be, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:51.331837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:51.332297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2566a-c6b1-4fc6-9fb8-9d43229a4848 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.333135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.333425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.333702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] PATH_INFO: `/v3/users/c96cbcb3dc994e45a66930828065645a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.335085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96cbcb3dc994e45a66930828065645a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:51.337669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.402018 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1036/2039] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:51 2022] PUT /identity/v3/projects/652de1d5c4614bbfaafe5df4628052b9/users/974ae63b8eb24baa9df89f38a5ad63be/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:51.420387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.420727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.421013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.507105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96cbcb3dc994e45a66930828065645a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.507732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96cbcb3dc994e45a66930828065645a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.512847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] MFA Rules not processed for user `974ae63b8eb24baa9df89f38a5ad63be`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:51.514478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:51.524271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96cbcb3dc994e45a66930828065645a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.541017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96cbcb3dc994e45a66930828065645a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.543874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96cbcb3dc994e45a66930828065645a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:51.592419 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1004/2040] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:51 2022] DELETE /identity/v3/users/c96cbcb3dc994e45a66930828065645a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:51.669707 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1037/2041] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:51.672034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:51.674400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:51.730671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:51.845249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53409645-3724-44d5-9de2-5e6758464cda admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:51.869164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:51.872382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:51.872843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:51.874079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:51.875834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:51.878792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:51.968955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1005/2042] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:51 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:51.973074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.049315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:52.113588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7863a-033e-449a-8d9f-a62d6681a5eb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.188379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.189711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.190159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.190787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.191936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:52.194109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:52.224128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1038/2043] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:51 2022] POST /identity/v3/users => generated 295 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:52.226147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.238802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968fb53-f35c-486c-9d8d-96ff952bbe0d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.244825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.245689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.246046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.247197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.249398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:52.251449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:52.266537 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1006/2044] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:52.270442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.286419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f27b5f-db44-4b9f-a384-57b4354ccf3b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.292743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.294217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.294625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.294967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] PATH_INFO: `/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc/users/80cabd8c672d41328827101afaaf2f1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.340745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8cf7bd750a74c02a06b70cba221d5bc, user_id=80cabd8c672d41328827101afaaf2f1e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:52.344268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:52.377065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1039/2045] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:52 2022] PUT /identity/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc/users/80cabd8c672d41328827101afaaf2f1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:52.378319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.396033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02213-f96c-4d64-8607-8239c721801f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.514255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.515046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.515389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.515727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.516656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:52.530724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff56e0-99e3-44d8-8398-7e044b01dad2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:52.548403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1007/2046] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:52.550939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.587812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc049c23-abe8-465f-b18b-a1f9f42a31a6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.600564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.601902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.602228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.602525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] PATH_INFO: `/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc/users/80cabd8c672d41328827101afaaf2f1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.604921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8cf7bd750a74c02a06b70cba221d5bc, user_id=80cabd8c672d41328827101afaaf2f1e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:52.610100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:52.634918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1040/2047] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:52 2022] PUT /identity/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc/users/80cabd8c672d41328827101afaaf2f1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:52.646731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.647016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.647387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.694463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] MFA Rules not processed for user `80cabd8c672d41328827101afaaf2f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:52.695489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:52.814289 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1008/2048] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:52.821549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.824766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:52.961272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636f9b6-220d-47f5-9271-1be12fd96848 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:52.981300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:52.982245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:52.982684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:52.983349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:52.989671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:52.992776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:53.009336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.012625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124fe51-7e6b-403c-ae1c-bcc7399a4115 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.032771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:53.060850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1041/2049] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:53.120798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.122049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.122826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.125025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1042/2050] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:50:53.131122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:53.150587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.164531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f8e36-b726-4066-ab92-875acc6e6179 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.171538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cabd8c672d41328827101afaaf2f1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cf7bd750a74c02a06b70cba221d5bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.172318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.172802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.173175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] PATH_INFO: `/v3/projects/652de1d5c4614bbfaafe5df4628052b9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.174124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc49eef-c309-49d6-965d-31a9b4cd9a18 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.174690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: Authorizing `identity:get_project(project_id=652de1d5c4614bbfaafe5df4628052b9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.176845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.179451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.180280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.180716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.181180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] PATH_INFO: `/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.182521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1158a2f81d4d5ba36d8b1462cc5df7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.184621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1043/2051] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:53 2022] GET /identity/v3/projects/652de1d5c4614bbfaafe5df4628052b9 => generated 377 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:50:53.185827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.232249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.257231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4b2f0-6506-475b-b682-b2b95d68bdcb tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.262307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1158a2f81d4d5ba36d8b1462cc5df7', 'request_id': 'req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:53.268663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:53.281139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1158a2f81d4d5ba36d8b1462cc5df7', 'request_id': 'req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:53.292358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1158a2f81d4d5ba36d8b1462cc5df7', 'request_id': 'req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:53.312979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] The token cache is being invalidate because project 8d1158a2f81d4d5ba36d8b1462cc5df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:53.313428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:53.316280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e9c9d29f-0cde-4ddb-b641-28c28813b9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:53.318761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1009/2052] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:52 2022] DELETE /identity/v3/projects/8d1158a2f81d4d5ba36d8b1462cc5df7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:53.325607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b71a14-9235-41cd-8426-3db4f806226d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.326167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b71a14-9235-41cd-8426-3db4f806226d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.326767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b71a14-9235-41cd-8426-3db4f806226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.355768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b0b71a14-9235-41cd-8426-3db4f806226d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:53.358489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b0b71a14-9235-41cd-8426-3db4f806226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:53.375133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.375875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.376671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.377008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.382553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:53.456205 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1010/2053] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:53.464289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.465174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.465744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.501361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:53.503158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:53.512738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.516457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.517647 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1011/2054] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:53.518799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.539711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba49bb-db7e-4ec1-baec-15def77f0fd0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.548015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1044/2055] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:53.565200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.566158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.566833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.567317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.568624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:53.570881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.572778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.606418 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1012/2056] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:53.608082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.631536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079c964-4c90-4d06-8415-f93b9be2e7e6 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.645693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.646352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.646667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.646945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.647598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.654842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.706577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1045/2057] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:53 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:53.709790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.722183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fe9cb-72c9-4abf-a28f-a062a3cc7da6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.751870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:53.808083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.809173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.809765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.810363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.811204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.814058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.849237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1013/2058] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:53 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:53.850439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.863090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f7d0c-adca-4493-8139-5c0fad87593f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.874205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.875432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.878665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.879155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.883737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.885822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.900027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1046/2059] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:53.906007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.919955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063046ca-9b5b-4425-b7d8-a13ad7733a86 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.925764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.926690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.927040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.927508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:53.928894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:53.932499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44ad1-a900-4ac6-8415-9be843ed3905 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:53.953633 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1014/2060] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:53.958222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:53.975273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadfd54-d383-46e5-98f4-b13684081591 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:53.982659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:53.983800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:53.984181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:53.985452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] PATH_INFO: `/v3/projects/c9e5d3e5b9bd4c599d4c59ec2e726eaf/users/8a030415ad574811ad4ec4fbb2babd0a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.005329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e5d3e5b9bd4c599d4c59ec2e726eaf, user_id=8a030415ad574811ad4ec4fbb2babd0a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.008446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba42c56-909c-4e37-8604-d58d933b93d0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.032043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1047/2061] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:53 2022] PUT /identity/v3/projects/c9e5d3e5b9bd4c599d4c59ec2e726eaf/users/8a030415ad574811ad4ec4fbb2babd0a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:54.044168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.044631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.045168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.058890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.060089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.060708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.062483 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1048/2062] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:50:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:50:54.069455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:50:54.071914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:54.124629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] MFA Rules not processed for user `8a030415ad574811ad4ec4fbb2babd0a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:54.128027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:54.220333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:54.292525 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1015/2063] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:54.298792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.316818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.321903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:54.522866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaa8a5-3dd2-41cc-abc4-911b4e475366 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.540359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9760312-20d2-41f3-a29c-476500666e5e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.541360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80cabd8c672d41328827101afaaf2f1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8cf7bd750a74c02a06b70cba221d5bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.543177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.543177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.550190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] PATH_INFO: `/v3/projects/652de1d5c4614bbfaafe5df4628052b9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.551118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: Authorizing `identity:get_project(project_id=652de1d5c4614bbfaafe5df4628052b9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.558194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.559196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.559699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.560143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.561476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.562193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.564065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1049/2064] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:50:54 2022] GET /identity/v3/projects/652de1d5c4614bbfaafe5df4628052b9 => generated 377 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:50:54.566862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.629263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1016/2065] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:54 2022] POST /identity/v3/projects => generated 387 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:54.632302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.637638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.647928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:54.655712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd889-3105-4998-b0e8-09b1c215cf44 tempest-FlavorsAccessTestJSON-641810292 tempest-FlavorsAccessTestJSON-641810292-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.696935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:54.770024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.770745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.771101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.771449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.772172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.772598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2c66-0292-49c4-bc05-33180bd9f932 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.777977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.779174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.780069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.780419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.780757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] PATH_INFO: `/v3/users/c7ff45029cc248b49b1ebafca987c6f8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.782000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ff45029cc248b49b1ebafca987c6f8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.784083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.823875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1050/2066] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:54 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:54.828173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.839143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7753b-023c-4b8c-9041-fe7940ec091a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.844898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.845513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.845796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.846069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.858540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.858540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.863268 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1051/2067] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:54.864420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.879073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32898e1e-4747-4b3b-a044-dd60a4c2e47d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:54.881304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ff45029cc248b49b1ebafca987c6f8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:54.882242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ff45029cc248b49b1ebafca987c6f8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:54.890926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:54.891976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:54.892313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:54.892596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] PATH_INFO: `/v3/projects/7ddb27a412004e398a4a5571f1894323/users/9d120ef590d44883b3a2cf3824137a71/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:54.896080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ff45029cc248b49b1ebafca987c6f8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:54.913914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ff45029cc248b49b1ebafca987c6f8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:54.915121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ff45029cc248b49b1ebafca987c6f8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:54.928950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ddb27a412004e398a4a5571f1894323, user_id=9d120ef590d44883b3a2cf3824137a71, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:54.934552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:54.952369 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1017/2068] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:54 2022] DELETE /identity/v3/users/c7ff45029cc248b49b1ebafca987c6f8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:54.960986 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1052/2069] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:54 2022] PUT /identity/v3/projects/7ddb27a412004e398a4a5571f1894323/users/9d120ef590d44883b3a2cf3824137a71/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:54.961998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:54.963486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.077291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.079749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.123861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af15340-04a5-424c-aee2-58dd60c06c90 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.144418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:55.189080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40ed6-92fd-4cd4-83b1-11ac5eaea806 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.203690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.204352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.204635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.204910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] PATH_INFO: `/v3/users/974ae63b8eb24baa9df89f38a5ad63be` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.209013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:55.209730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=974ae63b8eb24baa9df89f38a5ad63be)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.214329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.242872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.243645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.244036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.244196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.244841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.247050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.266409 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1018/2070] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:55.267496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.269649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.319084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ea3c4-694e-4d15-bf5a-7403383d0c59 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.324090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974ae63b8eb24baa9df89f38a5ad63be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.325116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974ae63b8eb24baa9df89f38a5ad63be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.347459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.348170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.348451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.348725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] PATH_INFO: `/v3/projects/a3ff52bec20740639122631bf53608ed` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.352045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974ae63b8eb24baa9df89f38a5ad63be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.356620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ff52bec20740639122631bf53608ed)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.360039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.395156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974ae63b8eb24baa9df89f38a5ad63be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.397081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974ae63b8eb24baa9df89f38a5ad63be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.418472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1053/2071] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:55 2022] DELETE /identity/v3/users/974ae63b8eb24baa9df89f38a5ad63be => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:55.419854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.421535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.443762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ff52bec20740639122631bf53608ed', 'request_id': 'req-b43db2d9-7082-484d-98ec-59a35251746d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.450668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ff52bec20740639122631bf53608ed', 'request_id': 'req-b43db2d9-7082-484d-98ec-59a35251746d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.462909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ff52bec20740639122631bf53608ed', 'request_id': 'req-b43db2d9-7082-484d-98ec-59a35251746d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.491155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] The token cache is being invalidate because project a3ff52bec20740639122631bf53608ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:55.491560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.492243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43db2d9-7082-484d-98ec-59a35251746d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.495180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1019/2072] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:55 2022] DELETE /identity/v3/projects/a3ff52bec20740639122631bf53608ed => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:55.498021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.498338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.499210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.526056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:55.527505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.533867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81689e-9f44-4a0e-9509-575a3a578db9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.537453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1020/2073] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:55.538181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.539485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.539599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.540383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.540669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.540959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] PATH_INFO: `/v3/projects/7ddb27a412004e398a4a5571f1894323/users/9d120ef590d44883b3a2cf3824137a71/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.543072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ddb27a412004e398a4a5571f1894323, user_id=9d120ef590d44883b3a2cf3824137a71, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.546806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.561096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84718740-a7a6-4584-9b64-2b328fbde889 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.571481 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1054/2074] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:55 2022] PUT /identity/v3/projects/7ddb27a412004e398a4a5571f1894323/users/9d120ef590d44883b3a2cf3824137a71/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:55.572490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.575635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.643606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.644269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.644547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.645523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] PATH_INFO: `/v3/users/dd4ed3b4807c47f7a1771e7d01ba3f97` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.647698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4ed3b4807c47f7a1771e7d01ba3f97)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.650022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.690260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71d35f-ff31-4156-9fc8-50212a05bea9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:55.696378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:55.697001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.697285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.697558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.708264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:55.710433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27850deb-456d-4b2a-9996-f4584f9e9574 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:55.712334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4ed3b4807c47f7a1771e7d01ba3f97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.713014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4ed3b4807c47f7a1771e7d01ba3f97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.714696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:55.723654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4ed3b4807c47f7a1771e7d01ba3f97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.725060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1055/2075] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:55.726656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:55.726954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:55.727760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:55.740336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:55.751269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4ed3b4807c47f7a1771e7d01ba3f97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.752148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4ed3b4807c47f7a1771e7d01ba3f97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:55.778669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] MFA Rules not processed for user `9d120ef590d44883b3a2cf3824137a71`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:55.780972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:55.786577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1021/2076] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:55 2022] DELETE /identity/v3/users/dd4ed3b4807c47f7a1771e7d01ba3f97 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:55.877430 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1056/2077] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:55.895662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:55.897525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.000427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee78e6-7c7b-450b-968e-ec6a6f339a8b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.012335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.013088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.013298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.013577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.046470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.049072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.062040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1022/2078] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:56.117381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.119100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.171452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.185888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c043d78-4a9e-4467-bc86-8c3c92365753 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.196384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.197006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.197287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.197587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.199965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.218101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632436f-932c-4781-ba79-20b439d0e862 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.225128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.225761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.226038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.226305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] PATH_INFO: `/v3/projects/9249e1a798344c07ac7f828954a3a1bb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.227513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9249e1a798344c07ac7f828954a3a1bb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.229955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.273835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.276065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.293306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1023/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:50:56.297160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9249e1a798344c07ac7f828954a3a1bb', 'request_id': 'req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.304085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9249e1a798344c07ac7f828954a3a1bb', 'request_id': 'req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.309869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.315371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.321879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9249e1a798344c07ac7f828954a3a1bb', 'request_id': 'req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.339166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa3f30-da40-467d-a8f3-39a75339e9ec service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.364952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] The token cache is being invalidate because project 9249e1a798344c07ac7f828954a3a1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:56.365372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.367419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc86e7b9-5d89-4cf2-9a4a-1a8b317c3c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.371375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1057/2080] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:56 2022] DELETE /identity/v3/projects/9249e1a798344c07ac7f828954a3a1bb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:56.376908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0bc8-cd5f-4636-859d-8ee8101ce0e1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.377204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0bc8-cd5f-4636-859d-8ee8101ce0e1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.377485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a0bc8-cd5f-4636-859d-8ee8101ce0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.419269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-697a0bc8-cd5f-4636-859d-8ee8101ce0e1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:56.420508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-697a0bc8-cd5f-4636-859d-8ee8101ce0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.421696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.422593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.422937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.423305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] PATH_INFO: `/v3/projects/652de1d5c4614bbfaafe5df4628052b9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.424495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652de1d5c4614bbfaafe5df4628052b9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.426673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.428570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:56.438472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1058/2081] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:56.442175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.442463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.442786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.468122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:56.476965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:56.483535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.504929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652de1d5c4614bbfaafe5df4628052b9', 'request_id': 'req-cab03478-bda3-4d4f-be83-cbf67736f901'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.518636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652de1d5c4614bbfaafe5df4628052b9', 'request_id': 'req-cab03478-bda3-4d4f-be83-cbf67736f901'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.527800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652de1d5c4614bbfaafe5df4628052b9', 'request_id': 'req-cab03478-bda3-4d4f-be83-cbf67736f901'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.541610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] The token cache is being invalidate because project 652de1d5c4614bbfaafe5df4628052b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:56.541900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.543493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.544889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1024/2082] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:56 2022] DELETE /identity/v3/projects/652de1d5c4614bbfaafe5df4628052b9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:56.548894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.550387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.580257 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1059/2083] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:56.586141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.603242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecbc45-9568-4418-adbf-0ffaa3369a8a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.609376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.610520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.610866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.611230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.611988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:56.612375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.616522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.645568 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1060/2084] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:56.650005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.664439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66e4a8-ca5b-4e3e-a452-ed1d3849951f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.672931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab03478-bda3-4d4f-be83-cbf67736f901 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.673118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.673974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.674323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.674692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.675579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.677793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.689198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.689964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.690630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.690922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] PATH_INFO: `/v3/users/80cabd8c672d41328827101afaaf2f1e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:56.693423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=80cabd8c672d41328827101afaaf2f1e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:56.695534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:56.733524 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1061/2085] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:56 2022] POST /identity/v3/projects => generated 359 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:56.737417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.779053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80cabd8c672d41328827101afaaf2f1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.780223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80cabd8c672d41328827101afaaf2f1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.797736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80cabd8c672d41328827101afaaf2f1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.817821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80cabd8c672d41328827101afaaf2f1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.818937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80cabd8c672d41328827101afaaf2f1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:56.855212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1025/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:56 2022] DELETE /identity/v3/users/80cabd8c672d41328827101afaaf2f1e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:56.882679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:56.885067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:56.917287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:56.923464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ef54c-8e37-4577-9369-f84e3ea0bb19 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:56.998478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de066983-cd73-4532-929c-483dba063958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:56.999130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066983-cd73-4532-929c-483dba063958 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:56.999412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066983-cd73-4532-929c-483dba063958 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:56.999684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066983-cd73-4532-929c-483dba063958 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.000457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de066983-cd73-4532-929c-483dba063958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.002269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de066983-cd73-4532-929c-483dba063958 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.008648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ef502-c176-4116-bee4-3a3d36b73e1a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.024411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.025381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.026323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.026891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] PATH_INFO: `/v3/users/8a030415ad574811ad4ec4fbb2babd0a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.029839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a030415ad574811ad4ec4fbb2babd0a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.032242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.034084 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1062/2087] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:56 2022] POST /identity/v3/users => generated 280 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:57.038127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de066983-cd73-4532-929c-483dba063958 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.041199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de066983-cd73-4532-929c-483dba063958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.056555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de066983-cd73-4532-929c-483dba063958 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.065267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.066637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.066967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.067202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.067896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.069675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.074609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:57.091334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1063/2088] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:57.092420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.104424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1c817-2f17-465a-b7ee-280e2cabcd59 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.113412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.114034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.114318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.114615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.115502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.117498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.119642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a030415ad574811ad4ec4fbb2babd0a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.121358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a030415ad574811ad4ec4fbb2babd0a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.135975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a030415ad574811ad4ec4fbb2babd0a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.144786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1064/2089] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:57.149480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.150543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a030415ad574811ad4ec4fbb2babd0a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.151324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53adeb29-9467-4853-b0d2-03682d0bbb91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a030415ad574811ad4ec4fbb2babd0a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.154820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.172456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091962ba-e20d-4631-b25b-43bb6434c7e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.176384 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1026/2090] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:56 2022] DELETE /identity/v3/users/8a030415ad574811ad4ec4fbb2babd0a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:57.178181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.178468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.179083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.207481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:57.208732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.246987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.247657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.247934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.248207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] PATH_INFO: `/v3/projects/7892c0f7b1b4446394ca03508f80d733/users/190b33a0d04b4d6c9ede6d4ee16a6cae/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.270448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7892c0f7b1b4446394ca03508f80d733, user_id=190b33a0d04b4d6c9ede6d4ee16a6cae, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.273985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.290472 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1027/2091] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:57.291792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.293545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.297383 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1065/2092] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:57 2022] PUT /identity/v3/projects/7892c0f7b1b4446394ca03508f80d733/users/190b33a0d04b4d6c9ede6d4ee16a6cae/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:57.301244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.304584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.314762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374330c-ff0c-46e5-812b-11b70f7ec1a6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.347122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:57.379956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc4cfa-2c16-4e0b-ba71-fc6b74ac42b3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.383733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.384340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.384682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.384924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.385334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.385954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.386312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.386775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] PATH_INFO: `/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.388095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8cf7bd750a74c02a06b70cba221d5bc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.390042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.400495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.401635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f721d4-d9c4-4664-aff7-a2aa3443d4c7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.416082 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1066/2093] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:57.418667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.419148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.420412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.450062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8cf7bd750a74c02a06b70cba221d5bc', 'request_id': 'req-c40180b8-c516-4a13-9f5a-403552d7c836'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.456495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] MFA Rules not processed for user `190b33a0d04b4d6c9ede6d4ee16a6cae`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:57.457717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.457833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8cf7bd750a74c02a06b70cba221d5bc', 'request_id': 'req-c40180b8-c516-4a13-9f5a-403552d7c836'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.467261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8cf7bd750a74c02a06b70cba221d5bc', 'request_id': 'req-c40180b8-c516-4a13-9f5a-403552d7c836'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.484508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] The token cache is being invalidate because project a8cf7bd750a74c02a06b70cba221d5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:57.484761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.485893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c40180b8-c516-4a13-9f5a-403552d7c836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:57.487461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1028/2094] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:57 2022] DELETE /identity/v3/projects/a8cf7bd750a74c02a06b70cba221d5bc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:57.493421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94a10-fd43-4d4a-b93a-07e2e639a8b3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.493643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94a10-fd43-4d4a-b93a-07e2e639a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.494138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd94a10-fd43-4d4a-b93a-07e2e639a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.535124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dcd94a10-fd43-4d4a-b93a-07e2e639a8b3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:57.537566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dcd94a10-fd43-4d4a-b93a-07e2e639a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.540696 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1067/2095] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:57.541744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.543826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.550201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:57.646680 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1029/2096] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:57.656458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.656980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.657398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.657754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf51319-7b1e-47e3-a526-e75d755e1b1f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.677669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.678605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.679064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.679525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.682185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.686429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:57.687921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.688090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.696893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1030/2097] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:57.701022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.711227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8e359-b04d-47e8-a53b-32f34c91f432 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.720568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.721186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.721475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.721738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.722603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:57.722909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.724517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.738550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1068/2098] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/projects => generated 361 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:57.740264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.748309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1031/2099] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:50:57.750203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.759940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027eee-550c-45aa-871b-40d2eb0bd2c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.802225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:57.822794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71139c-a573-45c5-9976-896cbf2e4cab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.823638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.824384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.825008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.825294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.825924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.827873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.828879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.829712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.830294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.830864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.831676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.835182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.861340 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1032/2100] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:57.863604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.865399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:57.867257 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1069/2101] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:57.868659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.882232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f5cc6-47d4-447c-b250-978ffad22b7b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.944071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44066dba-32cb-44ba-9531-fdcffa7be3cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.948367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.948738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:57.949554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.949835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.950026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:57.950127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.950655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:57.950783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.950935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] PATH_INFO: `/v3/projects/c9e5d3e5b9bd4c599d4c59ec2e726eaf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:57.952101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e5d3e5b9bd4c599d4c59ec2e726eaf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:57.953505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.954001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:57.977926 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1070/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:57 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:57.979077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:57.993497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e0da-acf2-4b93-82b0-cf94b5e63d65 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:57.997820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:58.009459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e5d3e5b9bd4c599d4c59ec2e726eaf', 'request_id': 'req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.017455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e5d3e5b9bd4c599d4c59ec2e726eaf', 'request_id': 'req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.027881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e5d3e5b9bd4c599d4c59ec2e726eaf', 'request_id': 'req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.046766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] The token cache is being invalidate because project c9e5d3e5b9bd4c599d4c59ec2e726eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:58.047446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.047930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.049412 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1033/2103] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:57 2022] DELETE /identity/v3/projects/c9e5d3e5b9bd4c599d4c59ec2e726eaf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:58.053329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.055216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.059680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.060295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.060573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.060843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.061483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.063123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.068269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3db75-1f1c-4a26-a399-d8faf9705ef3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.074427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.075452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.076324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.076607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.077285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.079257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.083640 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1071/2104] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:58.083640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.083640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.095651 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1034/2105] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:58.095651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.096932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.097744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed61616-0137-4f66-b991-502e46b94fcc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.103010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.103835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.104188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.104627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] PATH_INFO: `/v3/users/9d120ef590d44883b3a2cf3824137a71` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.105883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d120ef590d44883b3a2cf3824137a71)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.107923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.110513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bc35d-ed7e-47c3-bee1-e99a2b19019d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.118706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.119417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.119694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.120017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] PATH_INFO: `/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d/users/7cac33d820094e9c863b7bdf97bb30e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.138343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c03b645f72d4951bc3e03aeda3e5a3d, user_id=7cac33d820094e9c863b7bdf97bb30e9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.141073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.159421 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1035/2106] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:58 2022] PUT /identity/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d/users/7cac33d820094e9c863b7bdf97bb30e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:58.160290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.166611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d120ef590d44883b3a2cf3824137a71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.167453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d120ef590d44883b3a2cf3824137a71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.180730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d120ef590d44883b3a2cf3824137a71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.192202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:58.204357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d120ef590d44883b3a2cf3824137a71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.205182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d120ef590d44883b3a2cf3824137a71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.228937 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1072/2107] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:58 2022] DELETE /identity/v3/users/9d120ef590d44883b3a2cf3824137a71 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:58.229876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.231878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.252325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2359ef-7217-482c-a1c4-95e40e36a7a7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.342759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.343439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.343719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.343989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.345066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.347619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.366829 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1036/2108] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:58.368083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.370380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.388108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16620422-6ab4-43f3-a8a4-6cdaf9f71f6c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.400586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.400923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997b3a5-20d8-4890-94c4-2a8bf55f207f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.401630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.403574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.404897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.406841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.413081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.413656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.414912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.415196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.415471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] PATH_INFO: `/v3/projects/852c2aaefc8a480cb619a8ec990765cd/users/ed90b350882b4b04ba569228619f417c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.431232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1073/2109] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:58.435777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.440701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852c2aaefc8a480cb619a8ec990765cd, user_id=ed90b350882b4b04ba569228619f417c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.443769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee658c71-1b5e-408d-812c-e233039359b1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.448007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15423c9-389b-4eb6-93b6-6e39c4e3968a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.457231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.457994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.458276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.458597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] PATH_INFO: `/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d/users/7cac33d820094e9c863b7bdf97bb30e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.461275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c03b645f72d4951bc3e03aeda3e5a3d, user_id=7cac33d820094e9c863b7bdf97bb30e9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.464149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1037/2110] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:58 2022] PUT /identity/v3/projects/852c2aaefc8a480cb619a8ec990765cd/users/ed90b350882b4b04ba569228619f417c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:58.467827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99150c3-0e9b-44fa-872f-1d37531da727 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.477551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.478558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.479064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.479866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:58.501788 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1074/2111] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:58 2022] PUT /identity/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d/users/7cac33d820094e9c863b7bdf97bb30e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:58.504105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.504592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.504879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.544299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] MFA Rules not processed for user `ed90b350882b4b04ba569228619f417c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:58.545720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.546367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] MFA Rules not processed for user `7cac33d820094e9c863b7bdf97bb30e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:58.547786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.635166 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1038/2112] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:58.643181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.663924 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1075/2113] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:50:58.679266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.680450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.685442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:58.725190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8f24e-4831-4f2c-9117-3e8e754b1730 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.735134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.735772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.736027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.736299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.736940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.738947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.776524 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1039/2114] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:58 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:58.780767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.782421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.802021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a088d-647c-4984-8bb2-5909179e9cda None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.852905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df239b-aa6e-4bf5-8c97-d01391f7d24a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:58.858202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.858943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.859225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.859497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] PATH_INFO: `/v3/projects/7ddb27a412004e398a4a5571f1894323` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.860770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ddb27a412004e398a4a5571f1894323)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.862752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.890823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:58.891451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.891734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.892440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:58.921749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:58.921948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ddb27a412004e398a4a5571f1894323', 'request_id': 'req-097e55cc-717c-47e5-a477-2a7b71f074f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.935020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ddb27a412004e398a4a5571f1894323', 'request_id': 'req-097e55cc-717c-47e5-a477-2a7b71f074f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.946083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ddb27a412004e398a4a5571f1894323', 'request_id': 'req-097e55cc-717c-47e5-a477-2a7b71f074f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.963182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] The token cache is being invalidate because project 7ddb27a412004e398a4a5571f1894323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:50:58.963394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.964322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:58.967178 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1040/2115] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:50:58 2022] DELETE /identity/v3/projects/7ddb27a412004e398a4a5571f1894323 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:58.967946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:58.972189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:58.983911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:58.985576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef7a5-b4d7-4bfa-99ca-5bbbe7201ded service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:58.996227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1076/2116] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:50:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:50:58.997340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04274ed0-769b-4058-8861-cbb9fcda503e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:58.997626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04274ed0-769b-4058-8861-cbb9fcda503e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:58.997906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04274ed0-769b-4058-8861-cbb9fcda503e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.021642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-04274ed0-769b-4058-8861-cbb9fcda503e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:59.022961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-04274ed0-769b-4058-8861-cbb9fcda503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:59.049388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:59.073450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e55cc-717c-47e5-a477-2a7b71f074f6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.080367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.081262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.081403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.081722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.082425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.084777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.101831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1077/2117] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:59.112295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.112586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.112867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.117237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1041/2118] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:58 2022] POST /identity/v3/users => generated 303 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:59.118793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.136209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0167cf-ff21-4532-a648-97a5477d04bd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.143451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:59.143937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.145121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.145860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:59.145983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.146386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.147134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.149094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.156605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1078/2119] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:59.161533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.166414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1042/2120] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:59.170837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.176858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843f7e7-ea4c-4035-8f32-95a22cc7939b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.182598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.184012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.184201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36c4ba-fc73-483b-bf92-0d2eeb69f30f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.184308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.184545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.185200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:50:59.185507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.187280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.189709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.190581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.190971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.191575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] PATH_INFO: `/v3/projects/f22b3f66765049999747d2700736afe1/users/b2af2d5f141a4407bdfcb37a8f21a842/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.200162 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1079/2121] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:50:59.204497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.208729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22b3f66765049999747d2700736afe1, user_id=b2af2d5f141a4407bdfcb37a8f21a842, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.211657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.215504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c855600-71bc-4fcf-9ca2-16adf0ef9418 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.221316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.222085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.222225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.222453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.223152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.225037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.227561 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1043/2122] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:59 2022] PUT /identity/v3/projects/f22b3f66765049999747d2700736afe1/users/b2af2d5f141a4407bdfcb37a8f21a842/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:59.232422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.259992 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1080/2123] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:50:59.263653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.325660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be039f-f60b-469b-a8d4-6ee0c2f64423 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.340920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82572320-134c-42ed-87d0-ebf6cff94a70 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.347199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.347717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.347998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.348271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.348928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.351115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.383465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1081/2124] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:59.384519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.385671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.386431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.386841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.387121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.387765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.389530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.391749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:59.399599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6416a-c6ca-41a5-b25f-93c16fcfa33e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.402859 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1044/2125] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:59.405598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.406251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.406618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.406856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.407504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.408199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.410584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.423220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1082/2126] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:50:59.429476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140395f8-7b9a-4a2e-b6be-32ecd147174b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.436721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.450568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae1efd-3632-4470-b0aa-354408bacfde admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.455663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.456521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.457091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.457276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.457509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] PATH_INFO: `/v3/projects/f22b3f66765049999747d2700736afe1/users/b2af2d5f141a4407bdfcb37a8f21a842/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.458207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.458514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.458809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.459445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.460856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22b3f66765049999747d2700736afe1, user_id=b2af2d5f141a4407bdfcb37a8f21a842, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.461760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283576f-0921-4d41-a0e5-b01c5454f0ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.464138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.478708 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1045/2127] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:59 2022] PUT /identity/v3/projects/f22b3f66765049999747d2700736afe1/users/b2af2d5f141a4407bdfcb37a8f21a842/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:50:59.481971 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1083/2128] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:50:59.483130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.492585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a41c9-a2b6-42bc-bb9c-45731a67b2cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.492879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a41c9-a2b6-42bc-bb9c-45731a67b2cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.493159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a41c9-a2b6-42bc-bb9c-45731a67b2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.531938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-944a41c9-a2b6-42bc-bb9c-45731a67b2cb None None] MFA Rules not processed for user `b2af2d5f141a4407bdfcb37a8f21a842`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:59.533750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:59.534636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-944a41c9-a2b6-42bc-bb9c-45731a67b2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:59.618353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7827bce-f59a-48b4-9e62-d7c087ac0508 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.625393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.626111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.626421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.626890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] PATH_INFO: `/v3/projects/fc8cfbd9651040b1a4c9844f088a857b/users/a3590953b1ca4dcfb5da6a319bfb11fe/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.644106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc8cfbd9651040b1a4c9844f088a857b, user_id=a3590953b1ca4dcfb5da6a319bfb11fe, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.647459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.657381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1084/2129] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:59.664393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1046/2130] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:50:59 2022] PUT /identity/v3/projects/fc8cfbd9651040b1a4c9844f088a857b/users/a3590953b1ca4dcfb5da6a319bfb11fe/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:50:59.676514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.677077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.677077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.708576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] MFA Rules not processed for user `a3590953b1ca4dcfb5da6a319bfb11fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:50:59.709816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:59.779313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1085/2131] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:50:59.784328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.852634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae97eea-54ba-4d63-8655-9f7b0c737d67 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.857728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.858257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.858563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.858836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.859478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.861206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.861436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.863307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:50:59.888688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f358b30-d46d-44cd-bdad-d4b57c854ab5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:50:59.894021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:50:59.895191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:50:59.895581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:50:59.896017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] PATH_INFO: `/v3/users/190b33a0d04b4d6c9ede6d4ee16a6cae` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:50:59.897730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190b33a0d04b4d6c9ede6d4ee16a6cae)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:50:59.900485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:50:59.916508 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1047/2132] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:50:59 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:50:59.917887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:50:59.971998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190b33a0d04b4d6c9ede6d4ee16a6cae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:59.972848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190b33a0d04b4d6c9ede6d4ee16a6cae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:50:59.976136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:50:59.985060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190b33a0d04b4d6c9ede6d4ee16a6cae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.001914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190b33a0d04b4d6c9ede6d4ee16a6cae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.002677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190b33a0d04b4d6c9ede6d4ee16a6cae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.007996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8886-8cdd-4129-beb3-2bfabde03e00 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.013020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.013649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.013928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.014192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] PATH_INFO: `/v3/users/ed90b350882b4b04ba569228619f417c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.015372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed90b350882b4b04ba569228619f417c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.017308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.019815 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1086/2133] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:59 2022] DELETE /identity/v3/users/190b33a0d04b4d6c9ede6d4ee16a6cae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:00.020475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.022898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.087090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:00.098198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed90b350882b4b04ba569228619f417c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.099196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed90b350882b4b04ba569228619f417c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.108510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed90b350882b4b04ba569228619f417c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.113161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a4727-42c0-43bc-a20e-4ccf719305c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.117002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed90b350882b4b04ba569228619f417c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.117781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-784a0a80-6104-4867-b562-7c65ba897a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed90b350882b4b04ba569228619f417c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:00.118420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.118759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.119035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.119307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.119937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.121921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffacf7f0-d265-4bcb-9846-1baa53aed776 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.144458 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1048/2134] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:50:59 2022] DELETE /identity/v3/users/ed90b350882b4b04ba569228619f417c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:00.146086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1087/2135] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:00 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:00.147843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.148642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.150896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.151373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.151842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.154978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.186819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:00.188304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.188720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:00.190146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.280242 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1049/2136] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:00.280242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.281234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1088/2137] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:00.281435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.282150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.290391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.303943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149a0b5-b8ea-4a4f-a414-ada5afaf9cda None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.311497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.312018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.312207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.312575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.313212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.314949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.327511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1050/2138] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:00.328364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.330077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.370445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f56b41-e77b-4d5a-840c-769b1a5b714f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.375383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.375998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.376439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.376717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.390624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.392311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.398179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:00.406881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccbbfe-f0b1-4056-a5b7-39062b3920e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.411409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.412253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.412599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.412946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.423136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.424634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.426587 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1089/2139] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:00.428116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.437363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1051/2140] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:00.442076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cef755-bc7f-4e7e-b460-3d0b17b0be80 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.447474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.448323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.448606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.449394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] PATH_INFO: `/v3/projects/cd8a527890e94f4e841d702b8e766d17/users/71b1dc505d65433cb8cab4b8ff8480a1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.468231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8a527890e94f4e841d702b8e766d17, user_id=71b1dc505d65433cb8cab4b8ff8480a1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.472221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.491064 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1090/2141] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:00 2022] PUT /identity/v3/projects/cd8a527890e94f4e841d702b8e766d17/users/71b1dc505d65433cb8cab4b8ff8480a1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:00.495401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.507954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0d42-5356-4e35-a614-956898754486 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.576356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:00.611879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.612492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.612771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.613555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.614275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.615621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6f075-3b9d-4d35-8322-c2bb37bb3a98 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.650612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1052/2142] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:00.653748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.668287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fcd54-1669-4638-8afd-469a3f5ff8b0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.676360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.677028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.677377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.678035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] PATH_INFO: `/v3/projects/cd8a527890e94f4e841d702b8e766d17/users/71b1dc505d65433cb8cab4b8ff8480a1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.680692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8a527890e94f4e841d702b8e766d17, user_id=71b1dc505d65433cb8cab4b8ff8480a1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.685599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.701349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1091/2143] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:00 2022] PUT /identity/v3/projects/cd8a527890e94f4e841d702b8e766d17/users/71b1dc505d65433cb8cab4b8ff8480a1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:00.712588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.712877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.713222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.747904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] MFA Rules not processed for user `71b1dc505d65433cb8cab4b8ff8480a1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:00.749488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.845813 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1053/2144] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:00.846206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.848500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.855487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:00.858524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:00.964502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b080b6-cc1a-4ace-aa3a-a3ef30353e58 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.972666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.973831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.974739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.975168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] PATH_INFO: `/v3/projects/852c2aaefc8a480cb619a8ec990765cd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:00.976518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852c2aaefc8a480cb619a8ec990765cd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:00.979415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:00.986942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9b38-eb98-4129-a1e1-6a75711f2f63 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:00.993570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:00.994325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:00.994737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:00.995107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.006479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.008820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:01.019834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1054/2145] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:01.020754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:01.022460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.043825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36316a64-54a8-4f4b-ac34-71e890f6d79a service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:01.050481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852c2aaefc8a480cb619a8ec990765cd', 'request_id': 'req-5a427dee-6567-423f-9922-4fa520756244'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.056354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852c2aaefc8a480cb619a8ec990765cd', 'request_id': 'req-5a427dee-6567-423f-9922-4fa520756244'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.065064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852c2aaefc8a480cb619a8ec990765cd', 'request_id': 'req-5a427dee-6567-423f-9922-4fa520756244'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.098783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] The token cache is being invalidate because project 852c2aaefc8a480cb619a8ec990765cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:01.099256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.100491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.102378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1092/2146] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:00 2022] DELETE /identity/v3/projects/852c2aaefc8a480cb619a8ec990765cd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:01.106606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:01.108264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.132857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:01.150228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:01.151389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:01.153022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:01.153461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] PATH_INFO: `/v3/projects/7892c0f7b1b4446394ca03508f80d733` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.155053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7892c0f7b1b4446394ca03508f80d733)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.157928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:01.182998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:01.213329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a427dee-6567-423f-9922-4fa520756244 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:01.227112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7892c0f7b1b4446394ca03508f80d733', 'request_id': 'req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.246747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7892c0f7b1b4446394ca03508f80d733', 'request_id': 'req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.255545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7892c0f7b1b4446394ca03508f80d733', 'request_id': 'req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.269437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] The token cache is being invalidate because project 7892c0f7b1b4446394ca03508f80d733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:01.269732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.270399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.272455 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1055/2147] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:01 2022] DELETE /identity/v3/projects/7892c0f7b1b4446394ca03508f80d733 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:01.273305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:01.275652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.292625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:01.293469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:01.293749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:01.294017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.295941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.348088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45bba4-c72c-4ef7-8a9d-19b69a94afd6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:01.353144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:01.353951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:01.354119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:01.354483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] PATH_INFO: `/v3/users/b2af2d5f141a4407bdfcb37a8f21a842` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.356260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2af2d5f141a4407bdfcb37a8f21a842)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.358436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:01.360342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.362100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:01.373173 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1093/2148] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:01.374208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:01.375615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.390305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78113b96-38e2-497e-be5a-0255249d26e9 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:01.396043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:01.396773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:01.397153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:01.397523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] PATH_INFO: `/v3/users/7cac33d820094e9c863b7bdf97bb30e9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.401039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cac33d820094e9c863b7bdf97bb30e9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.402385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:01.447673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2af2d5f141a4407bdfcb37a8f21a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.450989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2af2d5f141a4407bdfcb37a8f21a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.480003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2af2d5f141a4407bdfcb37a8f21a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.484869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cac33d820094e9c863b7bdf97bb30e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.485804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cac33d820094e9c863b7bdf97bb30e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.497217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2af2d5f141a4407bdfcb37a8f21a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.498845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2af2d5f141a4407bdfcb37a8f21a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.499258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cac33d820094e9c863b7bdf97bb30e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.516002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cac33d820094e9c863b7bdf97bb30e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.519150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cac33d820094e9c863b7bdf97bb30e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:01.527484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1056/2149] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:01 2022] DELETE /identity/v3/users/b2af2d5f141a4407bdfcb37a8f21a842 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:01.537967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:01.545437 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1094/2150] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:01 2022] DELETE /identity/v3/users/7cac33d820094e9c863b7bdf97bb30e9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:01.763776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:01.765795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:01.775452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:01.860423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aee47-d18f-4860-8b01-41c9ee6bd08e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:01.966711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:01.967365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:01.967650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:01.967933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] PATH_INFO: `/v3/users/a3590953b1ca4dcfb5da6a319bfb11fe` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:01.969183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3590953b1ca4dcfb5da6a319bfb11fe)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:01.971268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.007815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.010793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.037900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3590953b1ca4dcfb5da6a319bfb11fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.038672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3590953b1ca4dcfb5da6a319bfb11fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.045085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a2d8-7029-49e6-aef8-0bbfbaddd022 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.051857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3590953b1ca4dcfb5da6a319bfb11fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.053556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.065964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.066593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.066879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] PATH_INFO: `/v3/projects/f22b3f66765049999747d2700736afe1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.068599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22b3f66765049999747d2700736afe1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.070560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.070736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3590953b1ca4dcfb5da6a319bfb11fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.071493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3590953b1ca4dcfb5da6a319bfb11fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.100119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1057/2151] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:01 2022] DELETE /identity/v3/users/a3590953b1ca4dcfb5da6a319bfb11fe => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:02.102780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.106113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.131337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22b3f66765049999747d2700736afe1', 'request_id': 'req-3fae4789-0cfe-47ac-a986-5c0097579f30'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.137735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22b3f66765049999747d2700736afe1', 'request_id': 'req-3fae4789-0cfe-47ac-a986-5c0097579f30'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.144989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22b3f66765049999747d2700736afe1', 'request_id': 'req-3fae4789-0cfe-47ac-a986-5c0097579f30'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.153599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:02.162010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] The token cache is being invalidate because project f22b3f66765049999747d2700736afe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:02.162308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.162989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fae4789-0cfe-47ac-a986-5c0097579f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.164676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1095/2152] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:02 2022] DELETE /identity/v3/projects/f22b3f66765049999747d2700736afe1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:02.170110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca06f2-7eaa-4739-9327-ba2875507e22 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.170110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca06f2-7eaa-4739-9327-ba2875507e22 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.170110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca06f2-7eaa-4739-9327-ba2875507e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.192251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-85ca06f2-7eaa-4739-9327-ba2875507e22 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:02.193631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-85ca06f2-7eaa-4739-9327-ba2875507e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.225149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e565ec-1ee7-4a85-ba03-907464383c39 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.289919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1096/2153] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:02.291243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.291525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.291793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.337015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:02.341934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.353676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.354331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.354645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.354972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] PATH_INFO: `/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.359591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c03b645f72d4951bc3e03aeda3e5a3d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.362415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1097/2154] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:02.363289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.365095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.367238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.456110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315d13f-316a-401f-a875-507ea91e6c95 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.460295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c03b645f72d4951bc3e03aeda3e5a3d', 'request_id': 'req-80bc5fd9-b895-47d1-8b58-ad781422aff3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.467982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c03b645f72d4951bc3e03aeda3e5a3d', 'request_id': 'req-80bc5fd9-b895-47d1-8b58-ad781422aff3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.475648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c03b645f72d4951bc3e03aeda3e5a3d', 'request_id': 'req-80bc5fd9-b895-47d1-8b58-ad781422aff3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.491928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] The token cache is being invalidate because project 3c03b645f72d4951bc3e03aeda3e5a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:02.492896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.494321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80bc5fd9-b895-47d1-8b58-ad781422aff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:02.497600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:02.500510 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1058/2155] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:02 2022] DELETE /identity/v3/projects/3c03b645f72d4951bc3e03aeda3e5a3d => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:02.502841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3eadba-46a0-4cd3-bcbd-d5ee4ea58097 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.503700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3eadba-46a0-4cd3-bcbd-d5ee4ea58097 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.504087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3eadba-46a0-4cd3-bcbd-d5ee4ea58097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.529682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3c3eadba-46a0-4cd3-bcbd-d5ee4ea58097 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:02.530942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c3eadba-46a0-4cd3-bcbd-d5ee4ea58097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.552638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.555146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.556534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.557778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.565049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.645638 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1059/2156] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:02.647335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.647335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.647583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.725108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:02.727265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.744637 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1060/2157] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:02.747176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.756922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.758232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2e4d7-d757-4b0b-ae0d-19fd1e77f4e1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.762940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:02.785360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1ce13-d5bc-44e0-acec-e4663368810b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.791267 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1098/2158] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:02.793793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.794250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.794603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.811945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.813138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.813425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.813747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.815626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:02.820161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:02.821452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:02.826134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.832013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1099/2159] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:02.835881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.837514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.848262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627818b-4111-49c3-a0fe-4ec712be5e32 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.853721 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1061/2160] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:02.854914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.856368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.856885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.857369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.858477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.858630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:02.859161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.862196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.875023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6594051-3237-40da-9104-e2dd1783c698 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.885240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1100/2161] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:51:02.890713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.891363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.891663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.891940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.892128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:02.892590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:02.897105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:02.929724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c58c5-39b5-4de2-9263-d6fb17ceb8da admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:02.945748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:02.947034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:02.948460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:02.985244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:02.986068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.003128 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1062/2162] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:03.004097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.008332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.078548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1101/2163] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:02 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:03.083183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.174567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c370d91-9ba0-4a2e-9e37-edf39049315f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.215912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35572a-1e0d-4030-911b-b3bc6cad4e36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.223850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.224590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.224878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.225345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.225990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.228417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.258690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1102/2164] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:03.260088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.263700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.269051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.269608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.270279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.270584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.271203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.273525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.283855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854966f6-2c19-43de-ae44-da88c04114c7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.289626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.290394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.290989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.290989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] PATH_INFO: `/v3/projects/fc8cfbd9651040b1a4c9844f088a857b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.292556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc8cfbd9651040b1a4c9844f088a857b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.294670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.300694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1063/2165] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:03.300694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.310884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e7cf-f0ff-4e8d-9d4d-23c44e9a6c70 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.315889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.316486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.316766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.317036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.317670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.319333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.329367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:03.331125 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1064/2166] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:03.332032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.343619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e93da-228a-4542-9ee7-cf5edb0f9e3d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.351320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc8cfbd9651040b1a4c9844f088a857b', 'request_id': 'req-10dbb932-c2b4-4222-8aec-5b410667088d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.357371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc8cfbd9651040b1a4c9844f088a857b', 'request_id': 'req-10dbb932-c2b4-4222-8aec-5b410667088d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.364251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc8cfbd9651040b1a4c9844f088a857b', 'request_id': 'req-10dbb932-c2b4-4222-8aec-5b410667088d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.377578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] The token cache is being invalidate because project fc8cfbd9651040b1a4c9844f088a857b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:03.377881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.378545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.382903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1103/2167] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:03 2022] DELETE /identity/v3/projects/fc8cfbd9651040b1a4c9844f088a857b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:03.382903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.382903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.397526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:03.417960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.418778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.419188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.419508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.420165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.422073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.433805 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1065/2168] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:03.434691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.438233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.452390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a11936-677a-491f-a7a2-d42925e2ecbf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.458682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.459790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.460079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.460354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] PATH_INFO: `/v3/users/71b1dc505d65433cb8cab4b8ff8480a1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.461838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b1dc505d65433cb8cab4b8ff8480a1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.465466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.485985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbb932-c2b4-4222-8aec-5b410667088d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.490933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.491579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.491914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.492193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.492860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.494709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.506292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1104/2169] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:03.507527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.509768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.529615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b1dc505d65433cb8cab4b8ff8480a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.530295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b1dc505d65433cb8cab4b8ff8480a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.533903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4aac-fcdd-4edf-b7d4-bd9c2b72950d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.541383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b1dc505d65433cb8cab4b8ff8480a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.542075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.542977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.543424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.543814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.545820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.548484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.555640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b1dc505d65433cb8cab4b8ff8480a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.556873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b1dc505d65433cb8cab4b8ff8480a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:03.562641 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1105/2170] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:03.563777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.566529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.582573 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1066/2171] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:03 2022] DELETE /identity/v3/users/71b1dc505d65433cb8cab4b8ff8480a1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:03.583781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.585444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.586399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4371f75-489c-4ba3-8345-a2164611c6c8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.668104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.668935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.669314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.669689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] PATH_INFO: `/v3/projects/5e68660c954f47f9bad1f7bee17f9435/users/9eec3a30bcc641ca8ecee19471007b1d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.687633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e68660c954f47f9bad1f7bee17f9435, user_id=9eec3a30bcc641ca8ecee19471007b1d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.689871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda781-d88d-4469-b7b1-0a6def69751c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.690617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9dc55-4278-43b9-8b06-fedf03d4f9ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.697224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.697938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.698373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.699310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] PATH_INFO: `/v3/projects/4ce082c6c630412088196dde1dfedeb1/users/99ddde6e7e7e4e8fbe67fe7f9fc6001d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.708394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1106/2172] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:03 2022] PUT /identity/v3/projects/5e68660c954f47f9bad1f7bee17f9435/users/9eec3a30bcc641ca8ecee19471007b1d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:03.716476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ce082c6c630412088196dde1dfedeb1, user_id=99ddde6e7e7e4e8fbe67fe7f9fc6001d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.720813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.720813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.721068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.723717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b44786-7a38-4544-8cb3-93ced3fc20d5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.740722 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1067/2173] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:03 2022] PUT /identity/v3/projects/4ce082c6c630412088196dde1dfedeb1/users/99ddde6e7e7e4e8fbe67fe7f9fc6001d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:03.751779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f57c96-5092-435a-866e-652310efd72a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.752414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f57c96-5092-435a-866e-652310efd72a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.752702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f57c96-5092-435a-866e-652310efd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.756657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] MFA Rules not processed for user `9eec3a30bcc641ca8ecee19471007b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:03.758118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.780463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:03.797505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74f57c96-5092-435a-866e-652310efd72a None None] MFA Rules not processed for user `99ddde6e7e7e4e8fbe67fe7f9fc6001d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:03.799266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74f57c96-5092-435a-866e-652310efd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:03.851969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:03.867000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1107/2174] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:03.872041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.894550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1068/2175] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:03.897219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f57c96-5092-435a-866e-652310efd72a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:03.974582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93978de5-b535-4bb2-ba68-85323b7d4b80 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.979644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.980806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.981087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.981437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.982178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.984154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:03.987620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f57c96-5092-435a-866e-652310efd72a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:03.993056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:03.993943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:03.994899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:03.995415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:03.997606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:03.999720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.014982 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1108/2176] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:04.019146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.034211 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1069/2177] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:03 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:04.035423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.099348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d5d3-c984-49c6-8751-ce34e694f4b6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.115848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bfb-4fec-4c43-89c2-a08ee1a25a35 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.121286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.121891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.122254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.122680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.123312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.125226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.150018 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1070/2178] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:04.150018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.150018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.155199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:04.162929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538dc17-b15e-435c-8606-0186a79ee91f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.168096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.168728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.169009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.169282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] PATH_INFO: `/v3/projects/cd8a527890e94f4e841d702b8e766d17` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.170404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8a527890e94f4e841d702b8e766d17)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.172438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.173671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.174367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.174694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.175046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.175672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.178538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.205231 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1109/2179] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:04.206183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.221360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4d4b1-4570-4b35-a9d2-c7386d7ae0bc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.226953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:04.231998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8a527890e94f4e841d702b8e766d17', 'request_id': 'req-93e4ce41-0089-406e-9ace-d3fa83d991de'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.240380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8a527890e94f4e841d702b8e766d17', 'request_id': 'req-93e4ce41-0089-406e-9ace-d3fa83d991de'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.249843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8a527890e94f4e841d702b8e766d17', 'request_id': 'req-93e4ce41-0089-406e-9ace-d3fa83d991de'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.268050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] The token cache is being invalidate because project cd8a527890e94f4e841d702b8e766d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:04.268369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.269018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.270752 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1071/2180] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:04 2022] DELETE /identity/v3/projects/cd8a527890e94f4e841d702b8e766d17 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.271453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.273205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.298896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.299713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.299981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.300367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.301078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.303247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f298a-6126-4d10-b4f1-3c3d80bc7843 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.317728 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1110/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:04.319839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.320129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.320399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.350007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:04.351431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.359590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1111/2182] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:04.360661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.361726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.364566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4ce41-0089-406e-9ace-d3fa83d991de admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.371909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.372652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.372943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.373308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.373983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.376325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6199e59-7484-4258-9726-49bc31d9838a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.377777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ad2a-cd45-4896-bed1-3636f5943e4b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.386995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.387671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.387949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.388222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] PATH_INFO: `/v3/projects/1883c633ee334e688fc777656675cad1/users/afe57f121621480b8d1ce9b209fca777/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.394908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1072/2183] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:04.394908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.394908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.394908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.406124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1883c633ee334e688fc777656675cad1, user_id=afe57f121621480b8d1ce9b209fca777, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.408909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.415147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:04.416450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.426292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1112/2184] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:04 2022] PUT /identity/v3/projects/1883c633ee334e688fc777656675cad1/users/afe57f121621480b8d1ce9b209fca777/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.427162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.428750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.483703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1073/2185] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:04.484788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.493964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a041-4d86-4bce-bfab-6e4063e37875 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.498176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b81af-5aa0-4b05-bc01-d25647c5a27f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.499282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.499915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.500189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.500453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] PATH_INFO: `/v3/projects/483f8a432244480698773200e1d49501/users/9d29ecd81aad409594eff4f3fb16edf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.502844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.503465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.503743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.504013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] PATH_INFO: `/v3/users/50d8a91794144d73a8942f5c3bac8cc8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.505139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d8a91794144d73a8942f5c3bac8cc8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.507115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.514870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483f8a432244480698773200e1d49501, user_id=9d29ecd81aad409594eff4f3fb16edf2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.517517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.531249 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1074/2186] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:04 2022] PUT /identity/v3/projects/483f8a432244480698773200e1d49501/users/9d29ecd81aad409594eff4f3fb16edf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.532117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.553877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:04.559833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d8a91794144d73a8942f5c3bac8cc8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.560947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d8a91794144d73a8942f5c3bac8cc8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.569866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d8a91794144d73a8942f5c3bac8cc8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.581576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:04.582069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d8a91794144d73a8942f5c3bac8cc8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.582800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d8a91794144d73a8942f5c3bac8cc8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:04.597948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1113/2187] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:04 2022] DELETE /identity/v3/users/50d8a91794144d73a8942f5c3bac8cc8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.598565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.599909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.604797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f1819-a757-4d74-81fb-7014622bb3d8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.693494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.694150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.695040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.695318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.696397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.697985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.698280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fddfb-138d-4441-9b30-13afaa6681a5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.703971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.704655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.704934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.705266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.705838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:04.706133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.707813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3694b-6e87-42f3-96ec-7addccd262b1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.710359 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1075/2188] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:04.714306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.720010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.722777 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1114/2189] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:04.724123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.724479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.724842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.736006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0913a46-9896-4712-b738-3b188e0dbebb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.743686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.744539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.746322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.746860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.747691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.749760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:04.750997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.751488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.763177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1115/2190] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:04.763177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.764620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.770290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1076/2191] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:04.771173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.781674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad9030-6d8b-450a-9200-a2a2d15178ca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.787075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879cf6-30ce-4b35-a944-531a4e9bfdee admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.788173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.789002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.789377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.789625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] PATH_INFO: `/v3/projects/1883c633ee334e688fc777656675cad1/users/afe57f121621480b8d1ce9b209fca777/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.792060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1883c633ee334e688fc777656675cad1, user_id=afe57f121621480b8d1ce9b209fca777, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.794429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.795237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.795611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.795725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.796001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.796701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.799033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.813187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1116/2192] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:04 2022] PUT /identity/v3/projects/1883c633ee334e688fc777656675cad1/users/afe57f121621480b8d1ce9b209fca777/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.815006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.820099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.838689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1077/2193] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/projects => generated 409 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:04.838689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.885202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee02c6-4b87-49a2-b9b5-aa2c5f9065bd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.916242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a838f3-cb28-4650-be0e-955e437fe8fe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:04.921739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.922383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.922683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.922950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] PATH_INFO: `/v3/projects/483f8a432244480698773200e1d49501/users/9d29ecd81aad409594eff4f3fb16edf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.924851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=483f8a432244480698773200e1d49501, user_id=9d29ecd81aad409594eff4f3fb16edf2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.927572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3d5f4-9f94-4272-a69a-be8d319dfecf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.931938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:04.938288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:04.939059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.939452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.940106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.940402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1078/2194] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:04 2022] PUT /identity/v3/projects/483f8a432244480698773200e1d49501/users/9d29ecd81aad409594eff4f3fb16edf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:04.941928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:04.942217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:04.942693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:04.959040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:04.960384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:04.971480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1117/2195] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:04.972409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:04.977253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] MFA Rules not processed for user `afe57f121621480b8d1ce9b209fca777`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:04.978693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:04.984618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109e59d-7dbb-4b76-9e35-ae614c4818d4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.030964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:05.055617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.056612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.057163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.057637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.059732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.061993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03fc69-7e74-4b45-9a6c-366f53108d96 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.088907 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1118/2196] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/users => generated 305 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:05.091310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.091790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.092094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.094864 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1079/2197] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:05.106688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.123790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3fb27-eb80-4df9-9821-32ed19241bbe None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.131410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.132028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.132505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.132614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.133189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.134843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.139468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] MFA Rules not processed for user `9d29ecd81aad409594eff4f3fb16edf2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:05.141856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.150924 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1080/2198] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:05.152367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.156723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.252606 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1119/2199] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:05.263912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.274458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe4af6-6dd2-4317-b945-9a82645312af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.280446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.280759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.281108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.281447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.282151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.284035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.291791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1c03-bcbd-4cf3-b408-82d086ec62b9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.294727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1120/2200] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:05.298406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.299057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.299341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.299761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.300309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.301787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.316735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.318402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecb9f3-8910-4a5d-929d-ee8b8b3394b4 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.330254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1081/2201] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:05.330813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79599-45f5-4c22-b887-9798f861916c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.331529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.331812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.332087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.336485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.337250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.337618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.337964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] PATH_INFO: `/v3/projects/c3800968cd944059a397e9c6b35f976d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.339849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3800968cd944059a397e9c6b35f976d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.341982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.353046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] MFA Rules not processed for user `9eec3a30bcc641ca8ecee19471007b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:05.354400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.397808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3800968cd944059a397e9c6b35f976d', 'request_id': 'req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.399484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:05.405809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3800968cd944059a397e9c6b35f976d', 'request_id': 'req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.416487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3800968cd944059a397e9c6b35f976d', 'request_id': 'req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.423139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1082/2202] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:05.424257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.434758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] The token cache is being invalidate because project c3800968cd944059a397e9c6b35f976d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:05.435046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.435798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.437610 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1121/2203] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:05 2022] DELETE /identity/v3/projects/c3800968cd944059a397e9c6b35f976d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:05.438262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301078c1-c4e8-456b-92db-a9658f450f75 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.441538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.443281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.492794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:05.502870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.503531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.503809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.504084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] PATH_INFO: `/v3/projects/57406e7d9d634bf980398bf9d3c1b044/users/c99668afe39143b6996b2051d8e5b2fe/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.518964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57406e7d9d634bf980398bf9d3c1b044, user_id=c99668afe39143b6996b2051d8e5b2fe, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.521661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.526015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3fd9a-55e9-4bb1-8540-3ec1e278d0e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.532588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.533189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.533463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.533732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.535230 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1083/2204] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:05 2022] PUT /identity/v3/projects/57406e7d9d634bf980398bf9d3c1b044/users/c99668afe39143b6996b2051d8e5b2fe/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:05.535634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.536026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.537705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.602295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb20974-26f9-4832-8b00-464e57bcc04b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.605613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.607057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.607219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.607683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.607962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.608245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] PATH_INFO: `/v3/users/17d98450c23947fdb9a196397eb45c61` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.609378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d98450c23947fdb9a196397eb45c61)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.611397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.617332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1122/2205] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:05.617804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.619837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.646973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22773c18-c273-4429-95e5-199c16a51184 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.668064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d98450c23947fdb9a196397eb45c61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.668809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d98450c23947fdb9a196397eb45c61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.677325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d98450c23947fdb9a196397eb45c61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.688652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d98450c23947fdb9a196397eb45c61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.689378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-37e01869-2375-4b27-9eda-8d6949995e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d98450c23947fdb9a196397eb45c61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:05.715796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1084/2206] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:05 2022] DELETE /identity/v3/users/17d98450c23947fdb9a196397eb45c61 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:05.715796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.715796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.715796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.725465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.726173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.726551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.726902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.728921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.736728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:05.747843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] MFA Rules not processed for user `c99668afe39143b6996b2051d8e5b2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:05.749563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.801820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:05.810759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.813216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:05.840165 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1085/2207] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:05 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:05.848444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:05.850749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:05.856055 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1123/2208] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:05.955790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff3a8e-8958-4884-bde9-7d0fd92e5240 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:05.962213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:05.964835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:05.964835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:05.964835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:05.964835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:05.966807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.005869 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1086/2209] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:05 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:06.006967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.018331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779ac14-2482-4651-a8ee-83f55307111c service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.084543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.087006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.090934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.091966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.092344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.092736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.093645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.097457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.104319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f2158-7edf-4e5e-b644-da477d16daaa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.110115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.110908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.111191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.111467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] PATH_INFO: `/v3/projects/b2477085569a4f058d27169122d5208f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.112708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2477085569a4f058d27169122d5208f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.115248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.129531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1124/2210] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/users => generated 311 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:06.132972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.146993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790094-71b4-4ac8-bb32-5d534a31d2ac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.157744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.158987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.159593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.159906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.160609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.162395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.179404 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1125/2211] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:06.197617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.202807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2477085569a4f058d27169122d5208f', 'request_id': 'req-18e4feb2-0164-43d6-94c8-1520cca936f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.209324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2477085569a4f058d27169122d5208f', 'request_id': 'req-18e4feb2-0164-43d6-94c8-1520cca936f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.213610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92b7ed-6df9-47d6-bc0e-3f0025cd60e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.228181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2477085569a4f058d27169122d5208f', 'request_id': 'req-18e4feb2-0164-43d6-94c8-1520cca936f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.242936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] The token cache is being invalidate because project b2477085569a4f058d27169122d5208f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:06.243225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.243877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18e4feb2-0164-43d6-94c8-1520cca936f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.246361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1087/2212] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:06 2022] DELETE /identity/v3/projects/b2477085569a4f058d27169122d5208f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:06.250858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ab0-cf31-4650-a497-09347727715f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.251220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ab0-cf31-4650-a497-09347727715f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.251561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87228ab0-cf31-4650-a497-09347727715f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.273931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-87228ab0-cf31-4650-a497-09347727715f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:06.275175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-87228ab0-cf31-4650-a497-09347727715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.279879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.280616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.280957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.281402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] PATH_INFO: `/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba/users/8c837d3b1ce04d5488fe8b522553388b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.284446 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1088/2213] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:06.293889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.294413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.294739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.299356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1bb9222e2a4ea2826c760a6e42a1ba, user_id=8c837d3b1ce04d5488fe8b522553388b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.302302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.320540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1126/2214] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:06 2022] PUT /identity/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba/users/8c837d3b1ce04d5488fe8b522553388b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:06.325412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.325778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:06.327254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.327629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.339714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:06.378835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:06.407590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1089/2215] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:06.415053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.425728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85c7b-f1a1-4ebb-8feb-09eaa5d8df82 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.427010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f0c14-dd5b-46fc-b661-84e05f8fedd1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.431668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.432257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.432505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.433098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.433526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.433643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.434145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.434490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.434811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.435670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:06.436117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.436650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.439716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.454620 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1127/2216] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:06.455514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.459674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1090/2217] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:06.460686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.467455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.476785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d74ee-1ce2-46ec-af34-5c6f9c3122b2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.482417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.483698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.483980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.484262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] PATH_INFO: `/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba/users/8c837d3b1ce04d5488fe8b522553388b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.486550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1bb9222e2a4ea2826c760a6e42a1ba, user_id=8c837d3b1ce04d5488fe8b522553388b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.494551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d4461-d5f2-44e0-8b32-8293b09972e8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.494886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.508650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.508940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.509705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.510117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] PATH_INFO: `/v3/users/99ddde6e7e7e4e8fbe67fe7f9fc6001d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.511968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99ddde6e7e7e4e8fbe67fe7f9fc6001d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.513912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.528208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1128/2218] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:06 2022] PUT /identity/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba/users/8c837d3b1ce04d5488fe8b522553388b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:06.528994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.617116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99ddde6e7e7e4e8fbe67fe7f9fc6001d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.617801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99ddde6e7e7e4e8fbe67fe7f9fc6001d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.629697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99ddde6e7e7e4e8fbe67fe7f9fc6001d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.649457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99ddde6e7e7e4e8fbe67fe7f9fc6001d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.650195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9e9c1eb6-3ca8-484b-961c-50e4ac4cde36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99ddde6e7e7e4e8fbe67fe7f9fc6001d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.661125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38474ce-958d-4870-80a1-ccb3ede6e669 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.666671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.667957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.668532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.668766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.669468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.671886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7299-da5c-4db2-b3c1-dc2322738990 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.675951 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1091/2219] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:06 2022] DELETE /identity/v3/users/99ddde6e7e7e4e8fbe67fe7f9fc6001d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:06.677288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e171e-6d95-470b-9333-ed3639aa6b55 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.677569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e171e-6d95-470b-9333-ed3639aa6b55 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.677839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e171e-6d95-470b-9333-ed3639aa6b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.700119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0f5e171e-6d95-470b-9333-ed3639aa6b55 None None] MFA Rules not processed for user `9eec3a30bcc641ca8ecee19471007b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:06.701575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0f5e171e-6d95-470b-9333-ed3639aa6b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.709699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1129/2220] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/projects => generated 395 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:06.709699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.709699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.709699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.743546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] MFA Rules not processed for user `8c837d3b1ce04d5488fe8b522553388b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:06.744938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.772362 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1092/2221] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:06.774379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.774908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.775380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.800687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:06.802313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.848972 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1130/2222] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:06.856466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.858738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.865795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:06.885878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:06.911744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1093/2223] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:06.911744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:06.911744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:06.924807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a2c2-97be-4e7a-95f6-9459f69dd651 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.930299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.931011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.931294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.931592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] PATH_INFO: `/v3/users/c99668afe39143b6996b2051d8e5b2fe` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.933383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99668afe39143b6996b2051d8e5b2fe)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.935659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.947368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86fe11-ca53-4055-a634-9517e24b4014 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:06.955930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:06.961082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:06.961563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:06.962485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:06.963345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:06.967398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:06.999057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99668afe39143b6996b2051d8e5b2fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:06.999748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99668afe39143b6996b2051d8e5b2fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:07.000694 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1131/2224] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:06 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:07.003551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.007235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:07.014708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99668afe39143b6996b2051d8e5b2fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:07.033295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99668afe39143b6996b2051d8e5b2fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:07.034907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99668afe39143b6996b2051d8e5b2fe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:07.077553 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1094/2225] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:06 2022] DELETE /identity/v3/users/c99668afe39143b6996b2051d8e5b2fe => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:07.081378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.086613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:07.110023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b5d63-e58e-41c4-88fe-e7cf6fccae89 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.176798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.177620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.177915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.179590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.180514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:07.216318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c687b-12ee-4ed6-b28c-b8c7f934eaa9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.224209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.225291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.225717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.225998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.235155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.238810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5282fcb-7bb3-4daa-a80e-a20883fe4d65 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.253652 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1095/2226] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:07.255175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.255469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.256636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.282665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:07.283968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:07.295482 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1096/2227] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:07.296982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.309921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8761e-8448-42f6-8f4a-47458dd58198 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.315045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.317049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.317191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.317740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.318022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.318472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.319947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.322148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.325857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:07.341800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1097/2228] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:07.342803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.356043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1132/2229] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:07.357031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.366646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ebf34-1244-4cd2-b2bc-aa52c6eaa7e1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.372504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.373265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.373711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.373996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.385118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.387016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245beb43-283e-4cb1-951e-a34ecfde2436 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.401026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1098/2230] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:07.419799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e19d2-d2bd-469f-88e6-5ed48d0486e9 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.429794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.430475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.430776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.431160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb/users/3f421dbad8b14e2186421e1278d3a875/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.472634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcc95e714424741acb548115f7c0acb, user_id=3f421dbad8b14e2186421e1278d3a875, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.481980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.514278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1133/2231] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:07 2022] PUT /identity/v3/projects/1dcc95e714424741acb548115f7c0acb/users/3f421dbad8b14e2186421e1278d3a875/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:07.528594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed1314-a781-46da-8225-a586498c5263 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.529043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed1314-a781-46da-8225-a586498c5263 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.529180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed1314-a781-46da-8225-a586498c5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.611786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-75ed1314-a781-46da-8225-a586498c5263 None None] MFA Rules not processed for user `3f421dbad8b14e2186421e1278d3a875`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:07.615868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-75ed1314-a781-46da-8225-a586498c5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:07.673243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:07.754207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1099/2232] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:07.766523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.866954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34def8-3a01-46bd-a5c8-da6d71d6025d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:07.873738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:07.874421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:07.874764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:07.875042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:07.875958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:07.877914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:07.922895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1134/2233] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:07 2022] POST /identity/v3/projects => generated 395 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:07.928798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed1314-a781-46da-8225-a586498c5263 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:07.943340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed1314-a781-46da-8225-a586498c5263 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.012168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.013932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.020534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.021488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.021773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.022047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.023117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.025609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.069934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1100/2234] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:07 2022] POST /identity/v3/users => generated 304 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:08.071182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.086090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912376d6-c236-4627-8993-7f17a2961fb5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.095048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.096491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.096960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.097556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.098688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.100826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.116692 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1101/2235] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:08.121931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.135419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616d057-345e-4710-8a2b-2a9463ab7c21 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.143991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ff19b-9500-41c2-9f5a-017fab96124f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.149232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.149886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.150172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.150442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] PATH_INFO: `/v3/projects/4ce082c6c630412088196dde1dfedeb1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.152251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.152412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ce082c6c630412088196dde1dfedeb1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.154251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.159755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.160013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.160296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] PATH_INFO: `/v3/projects/37b8d37bcac74b7996617c1561e96c19/users/830860e6e5aa438ca5e59691022fa2fa/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.181254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b8d37bcac74b7996617c1561e96c19, user_id=830860e6e5aa438ca5e59691022fa2fa, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.186066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.206712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1102/2236] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:08 2022] PUT /identity/v3/projects/37b8d37bcac74b7996617c1561e96c19/users/830860e6e5aa438ca5e59691022fa2fa/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:08.208061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.210121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.213696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ce082c6c630412088196dde1dfedeb1', 'request_id': 'req-55bfb786-6ef6-4abb-8903-d4c86eee12e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.219994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ce082c6c630412088196dde1dfedeb1', 'request_id': 'req-55bfb786-6ef6-4abb-8903-d4c86eee12e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.221929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:08.226648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ce082c6c630412088196dde1dfedeb1', 'request_id': 'req-55bfb786-6ef6-4abb-8903-d4c86eee12e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.244971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] The token cache is being invalidate because project 4ce082c6c630412088196dde1dfedeb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:08.245776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.246902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.248547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1135/2237] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:08 2022] DELETE /identity/v3/projects/4ce082c6c630412088196dde1dfedeb1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:08.249381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.250733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.285806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309011a3-f323-4cb4-8745-84d88ac2b99d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.303714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:08.319565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfb786-6ef6-4abb-8903-d4c86eee12e3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.325311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.325924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.326209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.326481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.327158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.328828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.339083 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1136/2238] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:08.340083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.342007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.350048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.350729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.351006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.351278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] PATH_INFO: `/v3/projects/57406e7d9d634bf980398bf9d3c1b044` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.352390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=57406e7d9d634bf980398bf9d3c1b044)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.354377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.355405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520daf2c-d429-479d-8caf-c4858169166b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.360077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.360697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.360984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.361256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] PATH_INFO: `/v3/users/afe57f121621480b8d1ce9b209fca777` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.362370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe57f121621480b8d1ce9b209fca777)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.364407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.404199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57406e7d9d634bf980398bf9d3c1b044', 'request_id': 'req-b708f4c4-8918-4564-911a-5b2cfa94f77f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.409767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57406e7d9d634bf980398bf9d3c1b044', 'request_id': 'req-b708f4c4-8918-4564-911a-5b2cfa94f77f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.420214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57406e7d9d634bf980398bf9d3c1b044', 'request_id': 'req-b708f4c4-8918-4564-911a-5b2cfa94f77f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.426734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe57f121621480b8d1ce9b209fca777'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.427439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe57f121621480b8d1ce9b209fca777'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.434229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] The token cache is being invalidate because project 57406e7d9d634bf980398bf9d3c1b044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:08.434547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.435497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.438888 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1103/2239] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:08 2022] DELETE /identity/v3/projects/57406e7d9d634bf980398bf9d3c1b044 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:08.438888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.439582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe57f121621480b8d1ce9b209fca777'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.442023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.459716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe57f121621480b8d1ce9b209fca777'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.461235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe57f121621480b8d1ce9b209fca777'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.479251 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1137/2240] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:08 2022] DELETE /identity/v3/users/afe57f121621480b8d1ce9b209fca777 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:08.483049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.486882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.538062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708f4c4-8918-4564-911a-5b2cfa94f77f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.616355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:08.621801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a555-5404-4185-9153-ce62e3d14bca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:08.625209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:08.638352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.639485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.640137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.640581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] PATH_INFO: `/v3/users/8c837d3b1ce04d5488fe8b522553388b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.640759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:08.642376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.642765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.643112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] PATH_INFO: `/v3/projects/37b8d37bcac74b7996617c1561e96c19/users/830860e6e5aa438ca5e59691022fa2fa/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.643534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c837d3b1ce04d5488fe8b522553388b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.645537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b8d37bcac74b7996617c1561e96c19, user_id=830860e6e5aa438ca5e59691022fa2fa, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:08.648059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.649567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f5456-38f4-4861-8131-8930891ad58a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:08.666031 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1104/2241] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:08 2022] PUT /identity/v3/projects/37b8d37bcac74b7996617c1561e96c19/users/830860e6e5aa438ca5e59691022fa2fa/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:08.668193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.668755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.669135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.698935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:08.700949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.725063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c837d3b1ce04d5488fe8b522553388b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.725777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c837d3b1ce04d5488fe8b522553388b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.741543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c837d3b1ce04d5488fe8b522553388b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.774074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c837d3b1ce04d5488fe8b522553388b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.774845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dd8f98b7-30fb-4e9f-bc51-d09e999822dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c837d3b1ce04d5488fe8b522553388b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:08.797938 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1138/2242] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:08 2022] DELETE /identity/v3/users/8c837d3b1ce04d5488fe8b522553388b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:08.803153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:08.803792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:08.804340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:08.873773 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1105/2243] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:08.875627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] MFA Rules not processed for user `830860e6e5aa438ca5e59691022fa2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:08.877109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:08.913735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:08.915440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:09.011680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1139/2244] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:09.013715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f526a-3ab1-44f1-bd11-d5443e649950 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.016715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.018475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:09.116263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.117114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.117628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.117836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.139712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22139ae4-38ea-4480-a7be-3d3cdf37652a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.147501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.148457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.148805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.149146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.150433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.167450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.186470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:09.187252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:09.217093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.219063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.236354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1106/2245] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:09.272470 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1140/2246] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:09 2022] POST /identity/v3/projects => generated 395 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:09.284916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.299407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4c27-9f0d-4abc-bbea-0cf13e267ad0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.467390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.468668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.469144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.469640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.470772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.476895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.537306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1107/2247] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:09 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:09.541378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.559857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d2cf3-d77c-4b08-8b68-4d75156ec703 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.571981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.572821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.577445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.578142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.578680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.580596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.630191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1141/2248] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:09.644035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.662933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23968a-bcc4-438c-bdcb-9f8283d686f4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.679467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.680184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.680975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.680975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.681461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.684247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4270c1-6ebf-4287-8c00-68afc5eab0f4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.706694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1108/2249] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:09.710946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.727719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096be57-75ed-4dcb-8ded-adab8a0527c9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:09.751888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:09.753817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.755475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.756676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] PATH_INFO: `/v3/projects/d64a82484b68429381efcbffd434ee1a/users/34e3097ce99c4de6adda3fa788c2cfe4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.802276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64a82484b68429381efcbffd434ee1a, user_id=34e3097ce99c4de6adda3fa788c2cfe4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:09.814252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:09.869518 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1142/2250] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:09 2022] PUT /identity/v3/projects/d64a82484b68429381efcbffd434ee1a/users/34e3097ce99c4de6adda3fa788c2cfe4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:09.889527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:09.889979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:09.891136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:09.936511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:09.938523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:09.955240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] MFA Rules not processed for user `34e3097ce99c4de6adda3fa788c2cfe4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:09.956821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.068694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1109/2251] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:09 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:10.069686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:10.071261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.106679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d895c9-3aa5-447a-b8d3-266a09084c84 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:10.113213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:10.114488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.114896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.115267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] PATH_INFO: `/v3/projects/1883c633ee334e688fc777656675cad1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.116533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1883c633ee334e688fc777656675cad1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:10.118754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:10.150321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:10.176340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444829f-2f54-4559-9e7d-3f0b7fdedef8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:10.211439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1883c633ee334e688fc777656675cad1', 'request_id': 'req-6fa94acb-1a1c-480b-bebf-92ad29350c47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.220118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1883c633ee334e688fc777656675cad1', 'request_id': 'req-6fa94acb-1a1c-480b-bebf-92ad29350c47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.227328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1883c633ee334e688fc777656675cad1', 'request_id': 'req-6fa94acb-1a1c-480b-bebf-92ad29350c47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.254436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] The token cache is being invalidate because project 1883c633ee334e688fc777656675cad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:10.254772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.257343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.258074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:10.258752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.259035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.259564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] PATH_INFO: `/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.262736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1bb9222e2a4ea2826c760a6e42a1ba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:10.264237 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1143/2252] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:09 2022] DELETE /identity/v3/projects/1883c633ee334e688fc777656675cad1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:10.264363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:10.265400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:10.268955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.356412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:10.423855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1bb9222e2a4ea2826c760a6e42a1ba', 'request_id': 'req-1bbf744f-bc29-47bb-acab-7bb38564a52b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.424725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94acb-1a1c-480b-bebf-92ad29350c47 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:10.449916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1bb9222e2a4ea2826c760a6e42a1ba', 'request_id': 'req-1bbf744f-bc29-47bb-acab-7bb38564a52b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.462285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1bb9222e2a4ea2826c760a6e42a1ba', 'request_id': 'req-1bbf744f-bc29-47bb-acab-7bb38564a52b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.494158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] The token cache is being invalidate because project 4e1bb9222e2a4ea2826c760a6e42a1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:10.494547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.497038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1bbf744f-bc29-47bb-acab-7bb38564a52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:10.502024 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1110/2253] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:10 2022] DELETE /identity/v3/projects/4e1bb9222e2a4ea2826c760a6e42a1ba => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:10.503999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb8408-f740-484e-a3f9-acd35873bdc5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.504613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb8408-f740-484e-a3f9-acd35873bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.504899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb8408-f740-484e-a3f9-acd35873bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.587911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:10.589476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.589758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.590031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.599158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.605587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b2fb8408-f740-484e-a3f9-acd35873bdc5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:10.613952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2fb8408-f740-484e-a3f9-acd35873bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.744286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:10.746069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711cae1d-adac-485e-b100-fcb82bf04334 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:10.775990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1111/2254] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:10.779566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fecd01-d6fa-4650-8d19-9284a604451e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.784025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fecd01-d6fa-4650-8d19-9284a604451e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.787241 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1144/2255] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:10.788267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fecd01-d6fa-4650-8d19-9284a604451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.788390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.791404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.791706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.845465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:10.847360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.859259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1145/2256] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:10.863256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:10.874541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74fecd01-d6fa-4650-8d19-9284a604451e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:10.878695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e544bf-376d-4bcb-8767-c5a454f99962 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:10.880456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74fecd01-d6fa-4650-8d19-9284a604451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:10.886846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:10.887133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.887372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.887908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.888908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:10.889032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:10.891303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:10.906953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1146/2257] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:51:10.919385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:10.924091 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1112/2258] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:10.935753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d8a6c-ddb6-4703-9486-af281b141bb3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:10.943430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880166aa-f03a-4089-b334-8b313a9a68ed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.943958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880166aa-f03a-4089-b334-8b313a9a68ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.944439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880166aa-f03a-4089-b334-8b313a9a68ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.965201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:10.965846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:10.966258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:10.966375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:10.967051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:10.969018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:10.978667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-880166aa-f03a-4089-b334-8b313a9a68ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:10.981450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-880166aa-f03a-4089-b334-8b313a9a68ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:11.001696 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1113/2259] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:11.001696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.001696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.001696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.025369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:11.026973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:11.042857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:11.154736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1114/2260] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:11.156033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.162927 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1147/2261] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:10 2022] POST /identity/v3/projects => generated 409 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:11.167005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.205810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e565dc-68f1-42c1-a11e-c94a3787becf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.240322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:11.286699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d9fd1-85c8-47e1-9c21-f02ee76ee517 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.290841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.291635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.292165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.292713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.293515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.298675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764732d-f1e7-4f78-914e-c1136b07b19d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.301704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.302376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.302736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.303010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.303669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:11.304421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.306815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.327513 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1115/2262] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:11.328602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.332757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:11.335548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1148/2263] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:11 2022] POST /identity/v3/users => generated 305 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:11.337752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e81b-f693-4e3f-9c78-d87daddd664b None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.338335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e81b-f693-4e3f-9c78-d87daddd664b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.338635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29e81b-f693-4e3f-9c78-d87daddd664b None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.341867 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1149/2264] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:11.342923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e81b-f693-4e3f-9c78-d87daddd664b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.363644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29e81b-f693-4e3f-9c78-d87daddd664b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.368584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.369180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.369460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.369733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.371071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.373396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.424327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1150/2265] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:11 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:11.425584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.437623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac32046-3f3d-4e00-a247-3737bd819b12 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.499499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff12198-ea90-473f-b655-0483c23a4c32 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.502712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:11.506848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.507786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.508481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.508481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.509325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.511645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.516049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.516866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.517419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.517902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.523586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1151/2266] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:11.524412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:11.526631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:11.583959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.585586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.597072 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1116/2267] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:51:11.597865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.609480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.609937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb1221-5f04-4a71-8764-63ae08b75515 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.615897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.616641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.616989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.617273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.617897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.620197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.670103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1117/2268] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:11 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:11.671123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.673306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:11.684167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e7f92-d394-4c99-9123-8b4b9bb52cf5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.690424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.691077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.691357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.691632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.692269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.694074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.709175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1118/2269] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:11.710117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.728869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba080f-d885-4148-ab39-fa93328b6139 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.732834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd467ec-9d5b-4d61-a140-cadb8540cfff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.740708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.741803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.742090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.742364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.744127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.746409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.748870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.749575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.749858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.750131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.751448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.753125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.754781 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1152/2270] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 395 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:11.755731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.773822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1119/2271] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:11.781911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.783346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229b63f-8fab-4397-9a84-0f5829ea2f8c tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.795574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.796239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.796515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.796785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] PATH_INFO: `/v3/projects/ab231b35cac14739857a122475926f75/users/c37bbe07404449939f9695f84a4a1138/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.799946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d795bd8-27c4-45be-8567-f3a4ff082c1f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.805520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.807162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.807162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.807162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.808132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.809977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ed7ff-c875-4b69-8870-664eec7d38ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.813920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab231b35cac14739857a122475926f75, user_id=c37bbe07404449939f9695f84a4a1138, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.816668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:11.821489 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1120/2272] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:11.822864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18ec6-9988-4445-b10e-66e2b204cabd None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.823691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18ec6-9988-4445-b10e-66e2b204cabd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.823969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c18ec6-9988-4445-b10e-66e2b204cabd None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.825324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1121/2273] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:11.826275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c18ec6-9988-4445-b10e-66e2b204cabd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.838281 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1153/2274] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:11 2022] PUT /identity/v3/projects/ab231b35cac14739857a122475926f75/users/c37bbe07404449939f9695f84a4a1138/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:11.839951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:11.841559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:11.841860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c18ec6-9988-4445-b10e-66e2b204cabd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:11.918588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:11.925464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:11.933161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:11.934755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] PATH_INFO: `/v3/projects/a8ecdacf8e4346119f4ddcc718cf0bfd/users/e65128ae545540cc927322247322ca03/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:11.936843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:11.973076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ecdacf8e4346119f4ddcc718cf0bfd, user_id=e65128ae545540cc927322247322ca03, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:11.981437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c0c0c-b65a-4829-afab-3ad4427b0632 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.016824 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1122/2275] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:11 2022] PUT /identity/v3/projects/a8ecdacf8e4346119f4ddcc718cf0bfd/users/e65128ae545540cc927322247322ca03/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:12.018755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd5b1-aec1-49b2-ad18-2252b3900145 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.018981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd5b1-aec1-49b2-ad18-2252b3900145 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.019258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfd5b1-aec1-49b2-ad18-2252b3900145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.059070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5edfd5b1-aec1-49b2-ad18-2252b3900145 None None] MFA Rules not processed for user `c37bbe07404449939f9695f84a4a1138`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:12.060384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5edfd5b1-aec1-49b2-ad18-2252b3900145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:12.083977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:12.133632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9ecf-5cc3-4731-8149-ad15460683a2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:12.152794 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1123/2276] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:12.152794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.152794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.152794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.186943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] MFA Rules not processed for user `e65128ae545540cc927322247322ca03`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:12.188084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:12.257085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:12.279352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:12.280003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.280806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.281089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.286386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:12.288578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.302539 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1154/2277] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:11 2022] GET /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 395 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:12.302539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:12.305296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1124/2278] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:12.315849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:12.463185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4589f5-8b66-45fe-9f6a-87658ce6929e tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:12.472283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:12.473941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.475455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.475886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.477315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:12.483247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.512693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3459725-df03-4614-8feb-dfc4a609d840 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:12.525527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:12.527073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.527795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.528278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.529326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:12.533110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.571709 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1155/2279] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/projects => generated 407 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:12.577085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:12.599777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1125/2280] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/projects => generated 377 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:12.605665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:12.783336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9742d5-81ed-42c4-903d-b1aa969670dd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:12.792904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:12.867801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc9c2-ac58-41d7-9fe1-4b6864053a6f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:12.884754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:12.886838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.887632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.888062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.896969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:12.901110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.901740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:12.904440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.904440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.904440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.904440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:12.907150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26516676-6a91-4881-a728-a4ab525b6188 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:12.961690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1156/2281] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/users => generated 310 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:12.961690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3b0c-96c3-45cc-b6c2-e2270371f83a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:12.961690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3b0c-96c3-45cc-b6c2-e2270371f83a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:12.962129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3b0c-96c3-45cc-b6c2-e2270371f83a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:12.976523 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1157/2282] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:12.978308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca3b0c-96c3-45cc-b6c2-e2270371f83a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:12.987524 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1126/2283] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:12 2022] POST /identity/v3/users => generated 295 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:12.988382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:13.009615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca3b0c-96c3-45cc-b6c2-e2270371f83a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.040959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.041611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.043053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.043053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.051457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.051981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.075565 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1158/2284] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:13.075565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.082455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:13.090882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fff786-4735-4765-b16f-4a9e99eb0b2f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.095888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.096493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.096809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.097085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.097720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.099778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.121131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1159/2285] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:13.124207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.155223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf91a-d8b2-4298-9163-432e1f3e6efe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.172280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.172521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.173188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.173472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] PATH_INFO: `/v3/projects/34df17012cfa4196bb99da22825c58c5/users/97a178801189432d831041a94f2d95b5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.222099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.241912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34df17012cfa4196bb99da22825c58c5, user_id=97a178801189432d831041a94f2d95b5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.264030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.288723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1160/2286] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:13 2022] PUT /identity/v3/projects/34df17012cfa4196bb99da22825c58c5/users/97a178801189432d831041a94f2d95b5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:13.289337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.359310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a6800-93ed-4d26-9d7d-ffffd5774d92 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.426049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07de5d-c448-4eef-9ee9-f223dcc66923 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.433907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.434772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.435154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.435529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] PATH_INFO: `/v3/projects/9491c85bbb9047dca561e4aa015e2db8/users/a9151c03b4a44fde8c9c0cf3785ee8df/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.455890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9491c85bbb9047dca561e4aa015e2db8, user_id=a9151c03b4a44fde8c9c0cf3785ee8df, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.458796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.493650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.494669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.495340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.495853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.501869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.504850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.507110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1127/2287] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:12 2022] GET /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 395 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:13.508730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.524466 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1161/2288] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:13 2022] PUT /identity/v3/projects/9491c85bbb9047dca561e4aa015e2db8/users/a9151c03b4a44fde8c9c0cf3785ee8df/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:13.531043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d45c-4f68-4506-a7f2-5bce31d89835 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.532089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.545156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:13.706176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.706916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.707194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.707468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.708113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.711714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.722177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcedf9-42e4-413b-99d2-c16582d8375b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.731057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.731758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.732782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.733257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.734612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.740682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.743734 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1128/2289] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:13.747428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.774985 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1162/2290] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:13.775650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ea968-3356-425b-afd6-da03563e6895 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.776644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:13.793153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.793153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.793153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.793153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] PATH_INFO: `/v3/projects/34df17012cfa4196bb99da22825c58c5/users/97a178801189432d831041a94f2d95b5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.801565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34df17012cfa4196bb99da22825c58c5, user_id=97a178801189432d831041a94f2d95b5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.801565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addf9ec-d279-4604-adc4-a29bfcd69e20 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.808432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b1d0d-2179-4daa-8888-7f8164b222e2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:13.818019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:13.819106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.819601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.820917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] PATH_INFO: `/v3/projects/9491c85bbb9047dca561e4aa015e2db8/users/a9151c03b4a44fde8c9c0cf3785ee8df/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.834782 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1129/2291] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:13 2022] PUT /identity/v3/projects/34df17012cfa4196bb99da22825c58c5/users/97a178801189432d831041a94f2d95b5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:13.837634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.837916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.838186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.847352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9491c85bbb9047dca561e4aa015e2db8, user_id=a9151c03b4a44fde8c9c0cf3785ee8df, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:13.850115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bf5ae-03b0-435a-9e0f-aab8ff463b5f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:13.880384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1163/2292] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:13 2022] PUT /identity/v3/projects/9491c85bbb9047dca561e4aa015e2db8/users/a9151c03b4a44fde8c9c0cf3785ee8df/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:13.893377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:13.893660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:13.893938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:13.896173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] MFA Rules not processed for user `97a178801189432d831041a94f2d95b5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:13.897779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:14.010416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] MFA Rules not processed for user `a9151c03b4a44fde8c9c0cf3785ee8df`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:14.015756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:14.092252 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1130/2293] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:14.094723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:14.220161 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1164/2294] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:14.287236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:14.471899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d80ec-d5f4-4985-a872-e2e0feb2d8c4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:14.477979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:14.480531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:14.481451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:14.481823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:14.482215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:14.497642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:14.499894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6757122-cdc4-4044-b233-224fdee6bd9b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:14.500219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:14.507197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:14.514329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1131/2295] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:14.516053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:14.516716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:14.517176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:14.517547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:14.527830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:14.529764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:14.555950 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1165/2296] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:14.609493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:14.632706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42724-9a37-4af8-b02c-a1bbd5693cc4 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:14.643522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:14.644458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:14.644684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:14.645112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:14.706396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:14.724512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4564b22-8648-4d2b-ad09-869dc0c14867 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:14.734022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:14.734819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:14.735097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:14.735361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:14.736790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:14.739053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:14.747665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:14.756445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1132/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:14.871698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:14.874852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269d7f3-4de2-4acb-b5b3-e71f135cccca service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:14.887610 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1166/2298] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:14 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:14.954193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:15.122208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3881e7-96c8-4ef5-b951-c8081f0f5bdf service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:15.156635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:15.159599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.160237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.161513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] PATH_INFO: `/v3/users/e65128ae545540cc927322247322ca03` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.164528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65128ae545540cc927322247322ca03)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:15.171112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:15.423808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65128ae545540cc927322247322ca03'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.425006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65128ae545540cc927322247322ca03'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.436605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65128ae545540cc927322247322ca03'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.453774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65128ae545540cc927322247322ca03'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.455068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65128ae545540cc927322247322ca03'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.505082 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1133/2299] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:14 2022] DELETE /identity/v3/users/e65128ae545540cc927322247322ca03 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:15.509987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.510799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.511556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.548699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:15.549933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:15.588265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:15.590466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:15.664257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:15.694433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1167/2300] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:15.723291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:15.727525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:15.809865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79fa1c-b04a-4473-8f9b-e879b71ebcaf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:15.817580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:15.818229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.818519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.818810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] PATH_INFO: `/v3/users/c37bbe07404449939f9695f84a4a1138` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.820122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37bbe07404449939f9695f84a4a1138)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:15.822319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:15.866404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:15.890854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37bbe07404449939f9695f84a4a1138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.891542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37bbe07404449939f9695f84a4a1138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.903683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37bbe07404449939f9695f84a4a1138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.910229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321506a-cf07-4249-8f12-dfc05b136106 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:15.916817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37bbe07404449939f9695f84a4a1138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.918122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f8f2ed78-533b-497b-a7bd-c7bd7a2ea81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37bbe07404449939f9695f84a4a1138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:15.926803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:15.930352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.931115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.933861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.940555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:15.948807 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1134/2301] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:15 2022] DELETE /identity/v3/users/c37bbe07404449939f9695f84a4a1138 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:15.950918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ded027-9201-45a8-8feb-61adb2cc1a60 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.951282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ded027-9201-45a8-8feb-61adb2cc1a60 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.952147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ded027-9201-45a8-8feb-61adb2cc1a60 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.953991 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1135/2302] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:15.955257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:15.955817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:15.955817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:15.968096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:15.969550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:15.980960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:15.983437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:16.008772 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1168/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:16.009962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:16.012372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:16.120865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1136/2304] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:16.154682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:16.156411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:16.185681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:16.189629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:16.315888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:16.381451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c327d-6811-48b2-bd90-e92c039af483 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:16.396970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:16.397597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:16.397876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:16.398408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:16.436359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:16.439486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0929323-c4af-4983-a489-9ba29d6cb9a9 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:16.456440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180b45-5b20-4b75-8c22-1f3899b5af69 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:16.474778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1137/2305] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:16.617236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:16.617890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:16.618170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:16.618443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:16.621414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:16.630298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:16.632608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1169/2306] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:16 2022] GET /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 395 bytes in 623 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:17.043973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.045075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.045578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.047543 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1138/2307] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:17.054015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:17.121050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:17.207058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:17.228005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663b976-f0f6-4048-a5a4-746694f29a25 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:17.234077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:17.234913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.235178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.235460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.236809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:17.239951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f828a-7474-41fb-8d7b-b0b87b7cd538 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:17.242012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1170/2308] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:17 2022] GET /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 395 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:17.391620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:17.393836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:17.426084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75be92d-33a5-4985-b8aa-61a3ffabb334 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:17.467438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.467997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.468129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:17.468487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.468706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.468993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.469272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] PATH_INFO: `/v3/projects/a8ecdacf8e4346119f4ddcc718cf0bfd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.470717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1171/2309] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:17.476823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ecdacf8e4346119f4ddcc718cf0bfd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:17.479477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:17.481113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:17.505488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:17.531920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af26b20-4370-42ab-8dad-79ff21f93da7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:17.548381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '830860e6e5aa438ca5e59691022fa2fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37b8d37bcac74b7996617c1561e96c19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:17.549707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.549877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.550468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] PATH_INFO: `/v3/projects/d64a82484b68429381efcbffd434ee1a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.554048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorizing `identity:get_project(project_id=d64a82484b68429381efcbffd434ee1a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:17.564167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:17.570341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1172/2310] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:17 2022] GET /identity/v3/projects/d64a82484b68429381efcbffd434ee1a => generated 395 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:17.639810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ecdacf8e4346119f4ddcc718cf0bfd', 'request_id': 'req-d183a614-27f2-4de6-b463-d4cd96a246a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.664802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ecdacf8e4346119f4ddcc718cf0bfd', 'request_id': 'req-d183a614-27f2-4de6-b463-d4cd96a246a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.673930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ecdacf8e4346119f4ddcc718cf0bfd', 'request_id': 'req-d183a614-27f2-4de6-b463-d4cd96a246a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.681345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:17.684551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:17.699839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] The token cache is being invalidate because project a8ecdacf8e4346119f4ddcc718cf0bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:17.700199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.703410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.705345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1139/2311] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:17 2022] DELETE /identity/v3/projects/a8ecdacf8e4346119f4ddcc718cf0bfd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:17.708886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:17.711668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:17.714231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be5e9-d8bb-4f48-8bd7-c06899913821 tempest-FlavorsAccessNegativeTestJSON-1295882863 tempest-FlavorsAccessNegativeTestJSON-1295882863-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:17.862193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183a614-27f2-4de6-b463-d4cd96a246a4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:17.868592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:17.870156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.870715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.871219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] PATH_INFO: `/v3/users/a9151c03b4a44fde8c9c0cf3785ee8df` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.873059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9151c03b4a44fde8c9c0cf3785ee8df)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:17.876742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:17.902077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:17.902780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:17.903061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:17.903334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] PATH_INFO: `/v3/projects/ab231b35cac14739857a122475926f75` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:17.911469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab231b35cac14739857a122475926f75)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:17.913428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:17.953164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9151c03b4a44fde8c9c0cf3785ee8df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.954521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9151c03b4a44fde8c9c0cf3785ee8df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.969492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9151c03b4a44fde8c9c0cf3785ee8df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.990600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9151c03b4a44fde8c9c0cf3785ee8df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.992593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9151c03b4a44fde8c9c0cf3785ee8df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:17.995824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:18.018073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab231b35cac14739857a122475926f75', 'request_id': 'req-67aa8fc5-80ed-4336-836c-f6c07f00bc55'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.034517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1140/2312] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:17 2022] DELETE /identity/v3/users/a9151c03b4a44fde8c9c0cf3785ee8df => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:18.035471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:18.037983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:18.048678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab231b35cac14739857a122475926f75', 'request_id': 'req-67aa8fc5-80ed-4336-836c-f6c07f00bc55'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.061323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab231b35cac14739857a122475926f75', 'request_id': 'req-67aa8fc5-80ed-4336-836c-f6c07f00bc55'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.084558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] The token cache is being invalidate because project ab231b35cac14739857a122475926f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:18.084558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.095000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.096653 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1173/2313] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:17 2022] DELETE /identity/v3/projects/ab231b35cac14739857a122475926f75 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:18.100401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:18.102406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:18.152858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79234b7-3c46-48c1-9d70-9ca9cdfb8685 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:18.159334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:18.160071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:18.160423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:18.160768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] PATH_INFO: `/v3/users/3f421dbad8b14e2186421e1278d3a875` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:18.162024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f421dbad8b14e2186421e1278d3a875)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:18.165085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:18.171259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:18.227448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa8fc5-80ed-4336-836c-f6c07f00bc55 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:18.247996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:18.250256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:18.252422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:18.254716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] PATH_INFO: `/v3/users/97a178801189432d831041a94f2d95b5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:18.258060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a178801189432d831041a94f2d95b5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:18.264591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:18.344621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f421dbad8b14e2186421e1278d3a875'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.346531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f421dbad8b14e2186421e1278d3a875'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.374151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f421dbad8b14e2186421e1278d3a875'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.398936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f421dbad8b14e2186421e1278d3a875'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.399919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f1792c8d-b3de-4689-b35e-e3c13bdc8108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f421dbad8b14e2186421e1278d3a875'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.486551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a178801189432d831041a94f2d95b5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.496455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a178801189432d831041a94f2d95b5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.503782 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1141/2314] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:18 2022] DELETE /identity/v3/users/3f421dbad8b14e2186421e1278d3a875 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:18.524435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:18.524975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:18.525171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:18.538804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a178801189432d831041a94f2d95b5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.607206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:18.609416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:18.614223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a178801189432d831041a94f2d95b5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.617271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a178801189432d831041a94f2d95b5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:18.666134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1174/2315] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:18 2022] DELETE /identity/v3/users/97a178801189432d831041a94f2d95b5 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:18.801293 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1142/2316] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:18.817215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:18.818744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:18.939581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c94a6-d253-4199-a701-651b64127653 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:18.949338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:18.950042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:18.950326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:18.950633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:19.082889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:19.084979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd0c10-90d6-41d2-9dd4-39139ec53ccb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:19.122577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1175/2317] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:19.311465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:19.313498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:19.320792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:19.348089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703db6ab-9ef4-4ced-8e3f-8423d59df772 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:19.375865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:19.377009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:19.379717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:19.380517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] PATH_INFO: `/v3/projects/9491c85bbb9047dca561e4aa015e2db8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:19.382933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9491c85bbb9047dca561e4aa015e2db8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:19.386349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:19.514233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9491c85bbb9047dca561e4aa015e2db8', 'request_id': 'req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:19.532274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9491c85bbb9047dca561e4aa015e2db8', 'request_id': 'req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:19.558013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9491c85bbb9047dca561e4aa015e2db8', 'request_id': 'req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:19.581356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] The token cache is being invalidate because project 9491c85bbb9047dca561e4aa015e2db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:19.581685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:19.583498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ddd2129b-3fa1-4c1a-ae1e-7174c169af76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:19.585914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1143/2318] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:19 2022] DELETE /identity/v3/projects/9491c85bbb9047dca561e4aa015e2db8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:19.591283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:19.591578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:19.592015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:19.636045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:19.644492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:19.652960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:19.850028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1176/2319] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:19.850554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:19.850950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:19.851388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:19.908757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:19.910139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:19.925595 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1144/2320] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:19.926949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:19.939410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6e5ab-b1dc-47d4-ac8a-617aa11f82c9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:19.945048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:19.945869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:19.946124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:19.946581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:19.947825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:19.947825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:19.950730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:19.984718 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1177/2321] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:51:19.985979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.010886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3488f-4073-48c0-abc1-632db3be4ec3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.023604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.024669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.025169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.025776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.028076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.035702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.100931 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1145/2322] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:19 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:20.105095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.121459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb551544-ce21-4944-82ac-365de8de6405 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.250421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.251407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.252044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.252533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.253284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.256584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.303494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1178/2323] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:20 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:20.308418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.332852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249e184-9816-40dd-bc0a-5e03542ac881 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.348547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.348547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.348547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.348547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.349413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.350702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.377801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1146/2324] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:20.378042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.394105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea63f-fcc9-40f3-987d-b58f781727d8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.404458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.405282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.405746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.406758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.408604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.410436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.450318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.452604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:20.463115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1179/2325] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:20.468022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.481983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987d3c-1100-465e-9b99-e67184a503a6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.517040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.518091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8c41b-728b-4bc9-a5ae-50d77e742ca7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.518861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.520358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.521209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] PATH_INFO: `/v3/projects/1dcc95e714424741acb548115f7c0acb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.524146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcc95e714424741acb548115f7c0acb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.530206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.531074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.531898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.533673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.534050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] PATH_INFO: `/v3/projects/1430428f54e54495a3546088d446026f/users/19537a4292714b70ba528489e1754fdd/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.593719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1430428f54e54495a3546088d446026f, user_id=19537a4292714b70ba528489e1754fdd, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:20.601655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c7c3f-8e29-472e-8ba2-807ede7145dd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:20.667507 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1180/2326] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:20 2022] PUT /identity/v3/projects/1430428f54e54495a3546088d446026f/users/19537a4292714b70ba528489e1754fdd/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:20.669285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.669570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.669844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.696220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcc95e714424741acb548115f7c0acb', 'request_id': 'req-ab759c52-247a-4ae5-89f1-7f5e805720f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:20.710768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcc95e714424741acb548115f7c0acb', 'request_id': 'req-ab759c52-247a-4ae5-89f1-7f5e805720f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:20.715362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] MFA Rules not processed for user `19537a4292714b70ba528489e1754fdd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:20.719594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:20.732556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcc95e714424741acb548115f7c0acb', 'request_id': 'req-ab759c52-247a-4ae5-89f1-7f5e805720f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:20.749880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] The token cache is being invalidate because project 1dcc95e714424741acb548115f7c0acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:20.750175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:20.751080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:20.755821 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1147/2327] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:20 2022] DELETE /identity/v3/projects/1dcc95e714424741acb548115f7c0acb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:20.759720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.762585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:20.801651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:20.837454 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1181/2328] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:20.841618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:20.862378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:20.867972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:20.977351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1edd3d-ac8b-4580-88f3-a8d8bcb252fb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.984321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:20.985409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:20.985804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:20.986357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:20.990566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab759c52-247a-4ae5-89f1-7f5e805720f2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:20.999263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.001414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:21.014331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:21.015110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1182/2329] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:21.016527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:21.017708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.017995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.018275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] PATH_INFO: `/v3/users/830860e6e5aa438ca5e59691022fa2fa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.018754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:21.027294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=830860e6e5aa438ca5e59691022fa2fa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.029278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:21.037378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eae7d-90fd-44e4-8981-18b259d70558 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:21.051713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:21.052417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.053401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.053790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] PATH_INFO: `/v3/projects/34df17012cfa4196bb99da22825c58c5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.055410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34df17012cfa4196bb99da22825c58c5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.058741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:21.157558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34df17012cfa4196bb99da22825c58c5', 'request_id': 'req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.176296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34df17012cfa4196bb99da22825c58c5', 'request_id': 'req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.207912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34df17012cfa4196bb99da22825c58c5', 'request_id': 'req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.216242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '830860e6e5aa438ca5e59691022fa2fa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.217356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '830860e6e5aa438ca5e59691022fa2fa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.234398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '830860e6e5aa438ca5e59691022fa2fa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.241443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] The token cache is being invalidate because project 34df17012cfa4196bb99da22825c58c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:21.241872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.242847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bfe85612-dd48-4f29-b84e-f6ee46a6a2e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.244952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1183/2330] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:21 2022] DELETE /identity/v3/projects/34df17012cfa4196bb99da22825c58c5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:21.249553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.250174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.250407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.269350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '830860e6e5aa438ca5e59691022fa2fa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.270294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b4aeeb5-e252-46ac-ba19-bde5d9a873cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '830860e6e5aa438ca5e59691022fa2fa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:21.318466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:21.319440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:21.336461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1148/2331] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:20 2022] DELETE /identity/v3/users/830860e6e5aa438ca5e59691022fa2fa => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:21.483089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:21.519005 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1184/2332] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:21.519770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.520323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.520807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.554721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:21.556289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:21.671299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1149/2333] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:21.675738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:21.688675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feb7f3-a2d9-4c6b-a761-94c3aeac582e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:21.696137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:21.697529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.698448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.699019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.700043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:21.700653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.702644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:21.721451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1185/2334] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:21.735265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:21.737017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b84735-38fe-4cae-97e5-3f786c078d52 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:21.742871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:21.743751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.744297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.744845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.746327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.748501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:21.817526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1150/2335] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:21 2022] POST /identity/v3/projects => generated 393 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:21.823221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:21.840861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0418fb5-0db7-4571-8009-6e05e301d467 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:21.977329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:21.978044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:21.981138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:21.981642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:21.982576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:21.991150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.064433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1186/2336] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:21 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:22.065030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.085282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3a409-8f17-4ffe-a45c-72a5e1032f60 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:22.129714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:22.130362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.133111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.133665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.134620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:22.142534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.171069 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1151/2337] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:51:22.172962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.187715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa60387-8a8e-44d0-9fc4-8c2b4ef228d7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:22.199255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:22.200399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.200949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.202123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.205031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:22.212065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1ea9f-5add-48a7-b397-c7178eea415a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.266410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1187/2338] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:22.267428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.288757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cc20-cfbe-45be-9681-9d2ad36c25a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:22.298004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:22.299008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.299138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.299353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] PATH_INFO: `/v3/projects/50a62390ba7b47feb78111804b7c5203/users/1b550836de5d42eab85965043d282782/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.319107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:22.333312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a62390ba7b47feb78111804b7c5203, user_id=1b550836de5d42eab85965043d282782, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:22.340879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.373677 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1152/2339] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:22 2022] PUT /identity/v3/projects/50a62390ba7b47feb78111804b7c5203/users/1b550836de5d42eab85965043d282782/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:22.387691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.388064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.388448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.422412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.424407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:22.488768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] MFA Rules not processed for user `1b550836de5d42eab85965043d282782`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:22.490250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:22.599827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309024f7-b56f-47bb-8fcc-9c6d707455b3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:22.605882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:22.607410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.607813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.608157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] PATH_INFO: `/v3/projects/37b8d37bcac74b7996617c1561e96c19` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.610334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b8d37bcac74b7996617c1561e96c19)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:22.612515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.688622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1188/2340] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:22.692838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.715508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1031d5f-c3fb-48c5-8302-013dd06c1fc1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:22.736406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b8d37bcac74b7996617c1561e96c19', 'request_id': 'req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:22.751370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b8d37bcac74b7996617c1561e96c19', 'request_id': 'req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:22.770190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b8d37bcac74b7996617c1561e96c19', 'request_id': 'req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:22.800606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] The token cache is being invalidate because project 37b8d37bcac74b7996617c1561e96c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:22.802534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:22.806834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:22.814236 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1153/2341] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:22 2022] DELETE /identity/v3/projects/37b8d37bcac74b7996617c1561e96c19 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:22.816324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.822788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:22.841574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:22.879705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:22.880777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:22.881212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:22.881594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:22.882403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:22.889213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:22.940817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1189/2342] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:22 2022] POST /identity/v3/projects => generated 395 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:22.946445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:22.950953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:23.047456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02718bef-8a4b-4dc1-bea2-7373a395986f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.053291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.054358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.055176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.055671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.056769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.059056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.073954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2918-bfa2-47f0-bb1c-4587ad548d50 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.091153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.091378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.091500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.091767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] PATH_INFO: `/v3/users/34e3097ce99c4de6adda3fa788c2cfe4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.094705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e3097ce99c4de6adda3fa788c2cfe4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.097080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.098540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1190/2343] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:22 2022] POST /identity/v3/users => generated 304 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:23.101977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.131201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9269-2701-468d-a7f8-badcf05c408d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.146826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.148940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.149440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.149933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.151315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.152962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:23.157001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.180440 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1191/2344] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:23.188231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.199906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e3097ce99c4de6adda3fa788c2cfe4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.201846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e3097ce99c4de6adda3fa788c2cfe4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.203276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b688c-461e-4971-a536-8f740a4faeb4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.209564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.210549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.211170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.211720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] PATH_INFO: `/v3/projects/bf434e8f97854da2a8c04d70dc0e4619/users/746925e822ce4240a6154e072cc04d22/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.216521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e3097ce99c4de6adda3fa788c2cfe4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.231239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e3097ce99c4de6adda3fa788c2cfe4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.233465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e3097ce99c4de6adda3fa788c2cfe4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.279880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf434e8f97854da2a8c04d70dc0e4619, user_id=746925e822ce4240a6154e072cc04d22, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.281150 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1154/2345] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:22 2022] DELETE /identity/v3/users/34e3097ce99c4de6adda3fa788c2cfe4 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:23.289318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.329215 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1192/2346] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:23 2022] PUT /identity/v3/projects/bf434e8f97854da2a8c04d70dc0e4619/users/746925e822ce4240a6154e072cc04d22/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:23.333105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.347688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:23.467503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.469346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:23.492075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4f5f6-dce8-4d36-b3c2-84faf9f68ee6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.566603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.567937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.568283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.568687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.569600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.571463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.587322 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1155/2347] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:23.588421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.618193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a296b0-1a3e-48d1-8c26-c07d6f2c6403 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.625885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.627088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.627088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.627304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] PATH_INFO: `/v3/projects/bf434e8f97854da2a8c04d70dc0e4619/users/746925e822ce4240a6154e072cc04d22/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.634604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48166b-dbaf-442e-8456-0889feff15f0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:23.635682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf434e8f97854da2a8c04d70dc0e4619, user_id=746925e822ce4240a6154e072cc04d22, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.638316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa763ffe-40bc-40e7-acba-f59dea0d7a54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.657673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:23.658548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.659189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.659470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] PATH_INFO: `/v3/users/19537a4292714b70ba528489e1754fdd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.661317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19537a4292714b70ba528489e1754fdd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:23.669323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:23.669574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1156/2348] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:23 2022] PUT /identity/v3/projects/bf434e8f97854da2a8c04d70dc0e4619/users/746925e822ce4240a6154e072cc04d22/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:23.679559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.679846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.680191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.749491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] MFA Rules not processed for user `746925e822ce4240a6154e072cc04d22`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:23.750770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:23.793090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:23.802879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:23.870668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19537a4292714b70ba528489e1754fdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.872277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19537a4292714b70ba528489e1754fdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.891044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19537a4292714b70ba528489e1754fdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.924833 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1157/2349] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:23.927966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19537a4292714b70ba528489e1754fdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.928701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdc62eed-4167-418e-bb7e-20e8d23c616f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19537a4292714b70ba528489e1754fdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:23.938784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:23.940233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:23.951384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1193/2350] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:23 2022] DELETE /identity/v3/users/19537a4292714b70ba528489e1754fdd => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:23.955640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:23.956144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:23.956610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:23.981413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:23.983256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.111069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a928c-935a-44f8-a33e-ec78e6372254 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.128676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1194/2351] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:24.129798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.131717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.154261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4044-8de5-41b9-a015-9cec09c9de89 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.160486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.161299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.161672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.162018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] PATH_INFO: `/v3/projects/d64a82484b68429381efcbffd434ee1a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.163360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64a82484b68429381efcbffd434ee1a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.165349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.198013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.198770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.199158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.199670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.239312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64a82484b68429381efcbffd434ee1a', 'request_id': 'req-0d264d41-1aa8-46da-84ae-c329db1f57b7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.247506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64a82484b68429381efcbffd434ee1a', 'request_id': 'req-0d264d41-1aa8-46da-84ae-c329db1f57b7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.254040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64a82484b68429381efcbffd434ee1a', 'request_id': 'req-0d264d41-1aa8-46da-84ae-c329db1f57b7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.271469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:24.275234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] The token cache is being invalidate because project d64a82484b68429381efcbffd434ee1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:24.277953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.277953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.284727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1195/2352] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:24 2022] DELETE /identity/v3/projects/d64a82484b68429381efcbffd434ee1a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:24.285769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.287461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.321861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.326813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5a374-74d4-408a-bb30-7dd05b142fa2 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.340140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:24.368547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1158/2353] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:24.371866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.372159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.372886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.385511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d264d41-1aa8-46da-84ae-c329db1f57b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.396090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.401502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.402109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.402524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.405680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.414943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:24.416325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.574357 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1159/2354] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:24.574953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.583315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.598918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e7e6-b920-4732-8d2f-57db8be68e94 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.606058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.606252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.606341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.606341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.606341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] PATH_INFO: `/v3/users/1b550836de5d42eab85965043d282782` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.606594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d368-be07-4a1d-b455-dfe257feab6c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.606667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b550836de5d42eab85965043d282782)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.608230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.618472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1196/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:24.621968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.622327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.622636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.658249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:24.661952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.676478 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1197/2356] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:24.697264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.720979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b550836de5d42eab85965043d282782'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.723930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b550836de5d42eab85965043d282782'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.724403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff1188-7739-444b-a70a-9ef048358e1a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.735364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.737121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.738412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.739135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.741557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:24.741960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.742288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b550836de5d42eab85965043d282782'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.747125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.761113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b550836de5d42eab85965043d282782'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.764677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e4c2b49f-8ef4-4002-a338-5ec6765a8d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b550836de5d42eab85965043d282782'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:24.794177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1198/2357] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:24.796505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.799622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:24.816876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2e52-847c-4768-a197-618b2c42ad98 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:24.833642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:24.834767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.835203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.835438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.835576 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1160/2358] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:24 2022] DELETE /identity/v3/users/1b550836de5d42eab85965043d282782 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:24.836211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:24.839707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:24.840303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:24.840833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:24.844785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:24.910085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1199/2359] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:24 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:24.918933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:24.921222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:24.922935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:25.111032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:25.136289 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1161/2360] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:25.155353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c686-d775-4722-91e2-339b16992ef4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:25.160455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:25.161548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.161739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.162105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.163371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:25.167006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:25.188687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:25.190245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:25.219871 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1200/2361] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:24 2022] POST /identity/v3/users => generated 291 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:25.232694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:25.268196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71601bd-d28f-4afd-8606-54da37819580 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:25.288432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:25.289176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.289575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.289951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.290736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:25.292481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:25.313841 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1201/2362] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:25.320410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:25.352436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ad7eb-fced-43e2-802a-acfff35adcb0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:25.370944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:25.371654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.372039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.372410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.373159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:25.374897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:25.404701 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1202/2363] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:25.408556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:25.423011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316b12-f55e-42a4-9233-1a647b048d62 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:25.426773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a30b2-ad12-43d6-a5fd-ae0c6f455040 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:25.443502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:25.444677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.445112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.445530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.464505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:25.501686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:25.503218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae75aa-a30d-43c7-87ec-fb2d1674c45e service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:25.506420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:25.509113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.511148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.512158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] PATH_INFO: `/v3/projects/5b7ec909a7664740a587e80121780359/users/855c05a3971a46b4a422141852c8e7c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.562100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1162/2364] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:25.664525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7ec909a7664740a587e80121780359, user_id=855c05a3971a46b4a422141852c8e7c0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:25.667445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:25.772386 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1203/2365] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:25 2022] PUT /identity/v3/projects/5b7ec909a7664740a587e80121780359/users/855c05a3971a46b4a422141852c8e7c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:25.786446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:25.786811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:25.787225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:25.888418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] MFA Rules not processed for user `855c05a3971a46b4a422141852c8e7c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:25.890011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:26.073367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1163/2366] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:26.080076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:26.253879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ca63-df33-456a-8b96-b8fc122565c2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:26.278531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:26.278531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.278531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.278531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.279659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:26.281044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:26.291009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:26.296090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:26.373199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56116e7-05ad-44b1-914a-63d66a1a7199 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:26.389027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:26.391423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.391597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.391856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] PATH_INFO: `/v3/projects/1430428f54e54495a3546088d446026f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.394718 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1204/2367] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:26 2022] POST /identity/v3/projects => generated 383 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:26.401591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:26.403341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1430428f54e54495a3546088d446026f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:26.409189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:26.560783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1430428f54e54495a3546088d446026f', 'request_id': 'req-1c73d962-53aa-482d-afb2-56f6d388931b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:26.583947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1430428f54e54495a3546088d446026f', 'request_id': 'req-1c73d962-53aa-482d-afb2-56f6d388931b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:26.605586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1430428f54e54495a3546088d446026f', 'request_id': 'req-1c73d962-53aa-482d-afb2-56f6d388931b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:26.610436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ce81-60fe-4a7e-92e3-fd562ce688dd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:26.633898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] The token cache is being invalidate because project 1430428f54e54495a3546088d446026f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:26.634193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:26.638548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c73d962-53aa-482d-afb2-56f6d388931b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:26.642466 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1164/2368] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:26 2022] DELETE /identity/v3/projects/1430428f54e54495a3546088d446026f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:26.649544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050371a-85b3-461b-ab74-70bffe46afcf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.650954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050371a-85b3-461b-ab74-70bffe46afcf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.651738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050371a-85b3-461b-ab74-70bffe46afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.740674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9050371a-85b3-461b-ab74-70bffe46afcf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:26.742283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9050371a-85b3-461b-ab74-70bffe46afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:26.765019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:26.865081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:26.866452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.867068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.867625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.871404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:26.878931 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1165/2369] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:26.879777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:26.887768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.888398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.888977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.928137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:26.930733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:26.933138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1205/2370] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:26 2022] POST /identity/v3/users => generated 298 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:26.934479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:26.946315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:26.956961 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1166/2371] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:26.957852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:26.963934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:26.980616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a43-a3cf-414a-8d78-50fd37455df7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:26.987287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503e7a3-e0f5-494e-a8d7-862ff93464bd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:26.995398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:26.996535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:26.996923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:26.997208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:26.997897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:26.999962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.002721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.003332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.003614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.004005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.004725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:27.005119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.010576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.039151 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1206/2372] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:27.042380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.045888 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1167/2373] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:27.049817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.072527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b856b7-7a69-4107-822f-7d39fde9de5e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.074325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa185c5-6f96-47cd-aa03-c4d54c23e0db admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.089640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.090568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.090856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.091134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.091784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.098567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.099025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.099673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.100140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.100797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] PATH_INFO: `/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451/users/d6908582909340dc8141903b041a53d5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.145165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9661139cfa4fb5a0019e7ac3cea451, user_id=d6908582909340dc8141903b041a53d5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.154640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.165901 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1168/2374] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:27 2022] POST /identity/v3/projects => generated 399 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:27.170084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.193910 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1207/2375] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:27 2022] PUT /identity/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451/users/d6908582909340dc8141903b041a53d5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:27.195173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.321255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb36e9-41c4-4f2e-9109-d45998129173 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.362522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6a60-d8ad-4077-a148-1bcbdfe9c586 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.372008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.372921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.373366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.373811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.374652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.376410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.397605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1208/2376] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:27.399498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.417061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2ae88-af9b-43f9-9e5f-db9d00cfdaa1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.423789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.424458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.424729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.425001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] PATH_INFO: `/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451/users/d6908582909340dc8141903b041a53d5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.427027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9661139cfa4fb5a0019e7ac3cea451, user_id=d6908582909340dc8141903b041a53d5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.431653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c3bbd-e873-442b-a7e1-ae5547a480a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.445046 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1209/2377] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:27 2022] PUT /identity/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451/users/d6908582909340dc8141903b041a53d5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:27.456790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.457124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.457336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.465169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.466394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.466937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.467432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.471795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.474061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.512122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] MFA Rules not processed for user `d6908582909340dc8141903b041a53d5`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:27.513380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:27.518065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1169/2378] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:27 2022] POST /identity/v3/users => generated 300 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:27.522064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.562981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252ffd7-eea2-4fc7-b205-7a85f9416c51 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.604953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:27.680653 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1210/2379] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:27.728116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.729009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.729392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.729672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.730327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.732495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.744744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1170/2380] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:51:27.748535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.765787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9982c-3db1-4c5a-be19-200d289a75ee None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.775818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.776809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.777290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.777757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.778649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.780779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.796079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1211/2381] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:27.799367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.815203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87c4ae-ac48-46f4-9212-3c4fb917f6c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.822866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:27.824524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.825024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.825551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] PATH_INFO: `/v3/projects/e175cb4032ec4e6aa900141abbaa04c4/users/6db52cb259fa426d860f9c9a48a149f3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.848056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e175cb4032ec4e6aa900141abbaa04c4, user_id=6db52cb259fa426d860f9c9a48a149f3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:27.854858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8d5a7-9538-460c-9b37-9b6de3d5f273 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:27.869108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:27.872811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:27.908676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb250ba-e82a-4d7c-b4b8-f5bedd3b92c2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:27.918185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1171/2382] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:27 2022] PUT /identity/v3/projects/e175cb4032ec4e6aa900141abbaa04c4/users/6db52cb259fa426d860f9c9a48a149f3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:27.923540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:27.923928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:27.926958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:27.985262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:28.023583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:28.024746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.108212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:28.109290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:28.109781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:28.110251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] PATH_INFO: `/v3/projects/50a62390ba7b47feb78111804b7c5203` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:28.112829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a62390ba7b47feb78111804b7c5203)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:28.116012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:28.210166 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1172/2383] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:28.224277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:28.225733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.232606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a62390ba7b47feb78111804b7c5203', 'request_id': 'req-d29478ca-1229-4c59-bb07-d814734681e7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.239905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a62390ba7b47feb78111804b7c5203', 'request_id': 'req-d29478ca-1229-4c59-bb07-d814734681e7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.248653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a62390ba7b47feb78111804b7c5203', 'request_id': 'req-d29478ca-1229-4c59-bb07-d814734681e7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.274305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] The token cache is being invalidate because project 50a62390ba7b47feb78111804b7c5203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:28.274305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.274977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.277195 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1212/2384] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:27 2022] DELETE /identity/v3/projects/50a62390ba7b47feb78111804b7c5203 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:28.278018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:28.279469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.354175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ace94-0bf8-4441-8b94-3fe5b743cbbb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:28.411099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29478ca-1229-4c59-bb07-d814734681e7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:28.419568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:28.420640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:28.420640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:28.420892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:28.422799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.458121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:28.491257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:28.492212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:28.492490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:28.492757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:28.500660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.529448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:28.569561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:28.575046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:28.604883 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1213/2385] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:28.606685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:28.612223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:28.703100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:28.705734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:28.737937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1173/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:28.797215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbe457-b445-479a-bc3a-12f0e76f3561 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:28.812607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:28.815421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:28.815974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:28.816826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] PATH_INFO: `/v3/users/746925e822ce4240a6154e072cc04d22` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:28.820205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746925e822ce4240a6154e072cc04d22)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:28.838261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:28.936713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746925e822ce4240a6154e072cc04d22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.937586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746925e822ce4240a6154e072cc04d22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:28.958375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746925e822ce4240a6154e072cc04d22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:29.003797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746925e822ce4240a6154e072cc04d22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:29.004888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746925e822ce4240a6154e072cc04d22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:29.039090 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1214/2387] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:28 2022] DELETE /identity/v3/users/746925e822ce4240a6154e072cc04d22 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:30.140320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:30.152281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c8a7-6892-4ada-b5cd-86855e7946c0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:30.174488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:30.180207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:30.347335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:30.347991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.348381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.348726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] PATH_INFO: `/v3/projects/bf434e8f97854da2a8c04d70dc0e4619` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:30.350962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf434e8f97854da2a8c04d70dc0e4619)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:30.363535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:30.449844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad50609-5b1c-4dc3-a1ef-b6d8903523f1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:30.465022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:30.465686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.465968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.466296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] PATH_INFO: `/v3/users/855c05a3971a46b4a422141852c8e7c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:30.470829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=855c05a3971a46b4a422141852c8e7c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:30.473717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:30.492419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:30.568447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf434e8f97854da2a8c04d70dc0e4619', 'request_id': 'req-f3e0080f-2c23-46aa-b4c4-d746ebf65265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.579856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf434e8f97854da2a8c04d70dc0e4619', 'request_id': 'req-f3e0080f-2c23-46aa-b4c4-d746ebf65265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.594190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf434e8f97854da2a8c04d70dc0e4619', 'request_id': 'req-f3e0080f-2c23-46aa-b4c4-d746ebf65265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.594379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855c05a3971a46b4a422141852c8e7c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.596221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855c05a3971a46b4a422141852c8e7c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.615430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855c05a3971a46b4a422141852c8e7c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.622130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] The token cache is being invalidate because project bf434e8f97854da2a8c04d70dc0e4619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:30.622813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.624019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3e0080f-2c23-46aa-b4c4-d746ebf65265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.626225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1174/2388] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:30 2022] DELETE /identity/v3/projects/bf434e8f97854da2a8c04d70dc0e4619 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:30.635691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507b76-3c8f-463f-ae10-856dc9470ef1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.635691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507b76-3c8f-463f-ae10-856dc9470ef1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.635691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66507b76-3c8f-463f-ae10-856dc9470ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:30.646364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855c05a3971a46b4a422141852c8e7c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.647110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50370738-f6e7-4d39-94a4-79605ce016e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855c05a3971a46b4a422141852c8e7c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:30.690482 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1215/2389] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:30 2022] DELETE /identity/v3/users/855c05a3971a46b4a422141852c8e7c0 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:30.694774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca07ea-0507-46ca-b56e-1f1a056904c2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.695097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-66507b76-3c8f-463f-ae10-856dc9470ef1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:30.696657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66507b76-3c8f-463f-ae10-856dc9470ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:30.698463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca07ea-0507-46ca-b56e-1f1a056904c2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.698789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca07ea-0507-46ca-b56e-1f1a056904c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:30.733485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:30.751190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9fca07ea-0507-46ca-b56e-1f1a056904c2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:30.752434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9fca07ea-0507-46ca-b56e-1f1a056904c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:30.905667 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1175/2390] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:30.909415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.909701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.909978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:30.969599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] MFA Rules not processed for user `9eec3a30bcc641ca8ecee19471007b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:30.973369 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1216/2391] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:30.975103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:30.975261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:30.975715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:30.976173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.001454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:31.003761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:31.025431 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1217/2392] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:31.029789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.059579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792a9ec-171f-4892-982a-95670ee92fda None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.080777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.083117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.083768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.084561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.086674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:31.087255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.096138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1176/2393] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:31.096954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.097091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.098544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:31.117079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1218/2394] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:31.122393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.145682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ff83-c23c-41fd-bbe9-7b32dbbdcea3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.154022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.154823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.155355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.155702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.157111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.160166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.230840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1219/2395] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:31 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:31.233971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.279821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653994c-2a29-4f0f-8314-d531232ff049 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.421675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:31.448767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.449912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.450354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.456888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.480131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.481841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.497445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a35b8e-7783-412c-a6e8-789397fa6941 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.513638 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1177/2396] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:31.526451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.530536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.530536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.530536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.534404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.537825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.599419 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1220/2397] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:31 2022] POST /identity/v3/users => generated 289 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:31.602649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.617468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b794ce2c-c191-4097-a988-2efcd65e24c0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.634176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.635015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.635447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.635718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.637592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.643929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.689534 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1178/2398] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:31.690581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.705664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89fb2-eca9-4da8-9ee4-f7bd5c0a9226 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.717434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.718324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.718662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.718939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.719592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.721212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53c1d-9441-4c34-ad79-64243e6c17ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.746997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1221/2399] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:31.750542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:31.767135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51108-7546-42b9-b331-fb1e4a087f97 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:31.787635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:31.788486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.788881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.789253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] PATH_INFO: `/v3/projects/d2124261003d42fda63ac1f75ecb84ae/users/f594da00255b413e92574deabe5b2427/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.809331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:31.848529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2124261003d42fda63ac1f75ecb84ae, user_id=f594da00255b413e92574deabe5b2427, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:31.855392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.855831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.856333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:31.875940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23dff3-600f-44ed-b786-4e6c5df119ad admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:31.901377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] MFA Rules not processed for user `9c03c67e0c4c4de7a186f28ac10d0b92`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:31.902787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:31.903267 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1179/2400] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:31 2022] PUT /identity/v3/projects/d2124261003d42fda63ac1f75ecb84ae/users/f594da00255b413e92574deabe5b2427/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:31.915088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:31.915524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:31.915872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:32.020510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] MFA Rules not processed for user `f594da00255b413e92574deabe5b2427`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:32.027749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:32.149766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1222/2401] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:32.232889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1180/2402] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:32.233636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:32.450272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428faf7a-9266-449f-a256-83f05b8c8c20 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:32.552464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:32.552464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:32.552464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:32.552464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:32.553186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:32.553186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:32.598945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1223/2403] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:32 2022] POST /identity/v3/projects => generated 377 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:32.607433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:32.658999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb496c1-3dee-4449-a76d-3c5d0189962f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:32.838269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:32.839811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:32.840159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:32.841583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:32.842245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:32.846802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:32.922884 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1181/2404] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:32 2022] POST /identity/v3/users => generated 295 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:32.926963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:32.939458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18f68-fca1-4bcf-b3a6-628f92799d90 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:32.953097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:32.953717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:32.954059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:32.954378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:32.955069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:32.958060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:32.983492 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1224/2405] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:32.987358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:33.013313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4558-1f01-433f-b711-33fbaed9cd16 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:33.033626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:33.036792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.037766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.039572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] PATH_INFO: `/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8/users/b01746ed5b6541b8b764e5b66c919193/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.111072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e088fce2b6b49ba90f68bf8ea1529f8, user_id=b01746ed5b6541b8b764e5b66c919193, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:33.120746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:33.168718 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1182/2406] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:32 2022] PUT /identity/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8/users/b01746ed5b6541b8b764e5b66c919193/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:33.174724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:33.195650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817c618-4ad8-424f-8d1c-c61204b2e177 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:33.201951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:33.204281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:33.354538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:33.365879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:33.399475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:33.401341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.401811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.402249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.403234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:33.407184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:33.416234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f6ca1-cfe7-49b2-ad41-e2a6ba250b17 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:33.432602 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1225/2407] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:33.436620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:33.437627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:33.438279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.438580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.439039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] PATH_INFO: `/v3/projects/5b7ec909a7664740a587e80121780359` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.444062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7ec909a7664740a587e80121780359)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:33.449623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:33.476767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf1387-2e11-4104-b43d-8fe387373790 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:33.496062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:33.496809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.497091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.497368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] PATH_INFO: `/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8/users/b01746ed5b6541b8b764e5b66c919193/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.501427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e088fce2b6b49ba90f68bf8ea1529f8, user_id=b01746ed5b6541b8b764e5b66c919193, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:33.504280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5279ba-2a12-4f4f-bd2f-8245675e3c75 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:33.546953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1226/2408] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:33 2022] PUT /identity/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8/users/b01746ed5b6541b8b764e5b66c919193/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:33.562308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.565714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.566213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.652533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7ec909a7664740a587e80121780359', 'request_id': 'req-705319e8-39af-4aa6-ad37-8582ea4b05e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:33.656238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] MFA Rules not processed for user `b01746ed5b6541b8b764e5b66c919193`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:33.659659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7ec909a7664740a587e80121780359', 'request_id': 'req-705319e8-39af-4aa6-ad37-8582ea4b05e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:33.660568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:33.671328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7ec909a7664740a587e80121780359', 'request_id': 'req-705319e8-39af-4aa6-ad37-8582ea4b05e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:33.709131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] The token cache is being invalidate because project 5b7ec909a7664740a587e80121780359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:33.710067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:33.715125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:33.717997 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1183/2409] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:33 2022] DELETE /identity/v3/projects/5b7ec909a7664740a587e80121780359 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:33.721690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:33.724159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:33.882801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1227/2410] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:33.896254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705319e8-39af-4aa6-ad37-8582ea4b05e0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:33.920747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:33.921401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:33.921705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:33.922270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] PATH_INFO: `/v3/users/d6908582909340dc8141903b041a53d5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:33.923145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6908582909340dc8141903b041a53d5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:33.925095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:34.047969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6908582909340dc8141903b041a53d5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:34.049926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6908582909340dc8141903b041a53d5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:34.081057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:34.081413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6908582909340dc8141903b041a53d5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:34.084177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:34.097140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6908582909340dc8141903b041a53d5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:34.098556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-25d6b0b4-9bac-4459-b398-f5101955faa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6908582909340dc8141903b041a53d5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:34.128492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:34.156924 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1184/2411] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:33 2022] DELETE /identity/v3/users/d6908582909340dc8141903b041a53d5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:34.247234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb2d5f-c777-4fdb-ab1a-84c0b85c0d75 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:34.369481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:34.458580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:34.459246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:34.460397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:34.460675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:34.466595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:34.679489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:34.679972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:34.680098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:34.684526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:34.686893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:34.712465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] MFA Rules not processed for user `93982876b1e24efb8ffa8c009532343a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:34.714186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:34.741318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1228/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:34.924786 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1185/2413] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:35.233183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:35.238928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:35.393554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:35.396632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:35.456073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6ed7-ff82-406a-98a9-b75b0918c1e9 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:35.465249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:35.467439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:35.467795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:35.470053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] PATH_INFO: `/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:35.472270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9661139cfa4fb5a0019e7ac3cea451)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:35.476345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:35.541829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9661139cfa4fb5a0019e7ac3cea451', 'request_id': 'req-1bf26c18-bfed-42ee-b458-b7f20be82fca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:35.566356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9661139cfa4fb5a0019e7ac3cea451', 'request_id': 'req-1bf26c18-bfed-42ee-b458-b7f20be82fca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:35.583674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9661139cfa4fb5a0019e7ac3cea451', 'request_id': 'req-1bf26c18-bfed-42ee-b458-b7f20be82fca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:35.616197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] The token cache is being invalidate because project eb9661139cfa4fb5a0019e7ac3cea451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:35.617097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:35.618232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1bf26c18-bfed-42ee-b458-b7f20be82fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:35.620613 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1229/2414] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:35 2022] DELETE /identity/v3/projects/eb9661139cfa4fb5a0019e7ac3cea451 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:35.627093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8de37-f1e4-403f-a853-2bb061ab5983 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:35.629276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8de37-f1e4-403f-a853-2bb061ab5983 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:35.629738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8de37-f1e4-403f-a853-2bb061ab5983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:35.673064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09459919-aafe-46d4-8797-76ec42dcbe5b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:35.683053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-58d8de37-f1e4-403f-a853-2bb061ab5983 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:35.687195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58d8de37-f1e4-403f-a853-2bb061ab5983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:35.705801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:35.832608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1230/2415] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:35.843474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:35.843964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:35.844292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:35.901458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:35.907165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:35.951482 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1231/2416] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:35.954061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:35.959663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:35.961027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:35.961589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:35.962008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] PATH_INFO: `/v3/users/f594da00255b413e92574deabe5b2427` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:35.965749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f594da00255b413e92574deabe5b2427)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:35.970746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.017958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1efbaa3-561d-4d9d-9790-520edee840b5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.037891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.039535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.040055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.042197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.043823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:36.044872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.047894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.089102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1232/2417] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:36.093155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.106429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35100d56-2e79-43a0-8fd7-ac44449c2fed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.112698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.113293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.113759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.114039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.114707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.116754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.140724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f594da00255b413e92574deabe5b2427'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:36.141673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f594da00255b413e92574deabe5b2427'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:36.144126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=92 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:36.154963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f594da00255b413e92574deabe5b2427'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:36.179678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f594da00255b413e92574deabe5b2427'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:36.182645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff0e1a5-154b-4649-842e-4996e8ff14b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f594da00255b413e92574deabe5b2427'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:36.185771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1233/2418] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:36 2022] POST /identity/v3/projects => generated 397 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:36.189483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.205221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:36.264332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1186/2419] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:35 2022] DELETE /identity/v3/users/f594da00255b413e92574deabe5b2427 => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:36.266818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.267104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.267376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.308296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:36.313161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:36.414330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21334-7baf-4880-9330-a639e6736a10 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.522100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1187/2420] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:36.579186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.579827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.580150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.580387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.581032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.583224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.609373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.610964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:36.613656 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1234/2421] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:36 2022] POST /identity/v3/users => generated 299 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:36.618192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.655074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690adb3-6aa4-4981-bbd5-de788099455b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.670269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.672654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.673099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.673778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.683718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.688762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.695183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:36.724882 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1235/2422] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:36.728713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.751463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b0e8-df19-4de8-99f2-2ca1c71e8c00 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.757860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.758521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.758840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.759746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.761853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ff320-6a49-44d7-90c9-a3e2e3f01e52 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.772936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.774526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.777532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.779833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.780628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.781201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.782605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.786622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2c27b-df0f-42aa-92e2-b60d16c8ace0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.787746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1188/2423] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:36.810479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1236/2424] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:36.814359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:36.838191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3f715-52a2-44a6-a028-d7bafc652b04 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:36.856948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:36.857635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.857917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.858191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] PATH_INFO: `/v3/projects/f3c583f2016e4edd94aeec7dd64f9673/users/ca78cf6d34e84f2cb5cb254c5d8af1da/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:36.917149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c583f2016e4edd94aeec7dd64f9673, user_id=ca78cf6d34e84f2cb5cb254c5d8af1da, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:36.920211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:36.954923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1189/2425] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:36 2022] PUT /identity/v3/projects/f3c583f2016e4edd94aeec7dd64f9673/users/ca78cf6d34e84f2cb5cb254c5d8af1da/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:36.960115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72847258-ec36-4449-931a-2eab109d8151 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:36.960748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72847258-ec36-4449-931a-2eab109d8151 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:36.961120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72847258-ec36-4449-931a-2eab109d8151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:37.025408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-72847258-ec36-4449-931a-2eab109d8151 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:37.038981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-72847258-ec36-4449-931a-2eab109d8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:37.191957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1237/2426] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:37.234833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:37.237942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:37.367637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:37.404671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bd82d-e8b6-4fb8-9a50-2ebde3468e94 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:37.428000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:37.431383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:37.431729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:37.432048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:37.461820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:37.464388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:37.478346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1190/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:37.997477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72847258-ec36-4449-931a-2eab109d8151 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:37.999019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-72847258-ec36-4449-931a-2eab109d8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:38.020275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72847258-ec36-4449-931a-2eab109d8151 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:38.027961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:38.028643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:38.029079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:38.029353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] PATH_INFO: `/v3/projects/d2124261003d42fda63ac1f75ecb84ae` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:38.031377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2124261003d42fda63ac1f75ecb84ae)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:38.034456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:38.105974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2124261003d42fda63ac1f75ecb84ae', 'request_id': 'req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.112990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2124261003d42fda63ac1f75ecb84ae', 'request_id': 'req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.121814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2124261003d42fda63ac1f75ecb84ae', 'request_id': 'req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.147886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] The token cache is being invalidate because project d2124261003d42fda63ac1f75ecb84ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:38.148548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.148896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.154342 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1238/2428] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:37 2022] DELETE /identity/v3/projects/d2124261003d42fda63ac1f75ecb84ae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:38.158747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:38.173953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fca28-e440-498e-ae65-3f91346e80b3 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:38.345474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:38.346140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:38.348489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:38.349006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] PATH_INFO: `/v3/users/b01746ed5b6541b8b764e5b66c919193` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:38.353446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01746ed5b6541b8b764e5b66c919193)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:38.360867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:38.495547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01746ed5b6541b8b764e5b66c919193'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.496520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01746ed5b6541b8b764e5b66c919193'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.520181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01746ed5b6541b8b764e5b66c919193'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.541185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01746ed5b6541b8b764e5b66c919193'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.542915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-419cd087-d904-4517-8e70-b79135dc9e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01746ed5b6541b8b764e5b66c919193'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:38.590660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1191/2429] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:38 2022] DELETE /identity/v3/users/b01746ed5b6541b8b764e5b66c919193 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:40.755934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:40.759659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:40.781667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a01f9-34be-4c4b-a846-65d20d5eb03c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:40.890771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:40.891593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:40.891879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:40.892155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] PATH_INFO: `/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:40.893734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e088fce2b6b49ba90f68bf8ea1529f8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:40.901915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:40.916621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:40.968896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e088fce2b6b49ba90f68bf8ea1529f8', 'request_id': 'req-a8d0aa3f-1251-4539-9384-f387fe39246c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:40.987374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e088fce2b6b49ba90f68bf8ea1529f8', 'request_id': 'req-a8d0aa3f-1251-4539-9384-f387fe39246c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:41.006969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e088fce2b6b49ba90f68bf8ea1529f8', 'request_id': 'req-a8d0aa3f-1251-4539-9384-f387fe39246c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:41.037254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] The token cache is being invalidate because project 4e088fce2b6b49ba90f68bf8ea1529f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:41.037770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:41.038858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a8d0aa3f-1251-4539-9384-f387fe39246c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:41.041630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1239/2430] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:40 2022] DELETE /identity/v3/projects/4e088fce2b6b49ba90f68bf8ea1529f8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:41.046356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.046914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.046914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.083661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:41.086920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:41.213767 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1192/2431] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:41.222398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15994191-2b8d-405c-81bc-85937659f962 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.223022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15994191-2b8d-405c-81bc-85937659f962 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.223486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15994191-2b8d-405c-81bc-85937659f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.258210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-15994191-2b8d-405c-81bc-85937659f962 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:41.261046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15994191-2b8d-405c-81bc-85937659f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:41.286825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1240/2432] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:41.291184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.311403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75191-7227-4984-95c2-6b91d6777677 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.326096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.326954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.327413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.327838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.328685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:41.329147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.331235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.334246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:41.354338 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1193/2433] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:41.358465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15994191-2b8d-405c-81bc-85937659f962 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.375642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15994191-2b8d-405c-81bc-85937659f962 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.383819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.386917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.387374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.387828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.389025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.392261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.453033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1241/2434] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:41 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:41.458454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.490318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51ae4d-c5e2-452f-908b-af777329b439 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.610567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.611349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.611739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.613022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.614196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.618887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.679284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1194/2435] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:41 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:41.683509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.706083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b635e-0642-47ca-814e-677ca8100469 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.711097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.711717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.712201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.712513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.713171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.714825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.734554 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1242/2436] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:41.738753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.750343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bb0f9-fdca-4121-839d-9a6a6342aecf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.759300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.760101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.760505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.761075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.761822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.763924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2d39e-c74b-4521-a0be-3714cdd9b9ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.781804 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1195/2437] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:41.783638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:41.799142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628be13b-0dc2-41ef-ba65-e10213f7b64b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:41.806768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:41.808073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.808452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.808870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab/users/8c3a535cc471436689e916bd1cbc3159/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:41.858347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43a52b503c946659d54675548a49bab, user_id=8c3a535cc471436689e916bd1cbc3159, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:41.863290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15aa26-0c06-48ca-bc6a-1cff44165cc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:41.896948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1243/2438] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:41 2022] PUT /identity/v3/projects/e43a52b503c946659d54675548a49bab/users/8c3a535cc471436689e916bd1cbc3159/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:41.913358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:41.913642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:41.914111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.009357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] MFA Rules not processed for user `8c3a535cc471436689e916bd1cbc3159`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:42.010773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:42.080763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4869-d210-4761-8ef5-ed3d8fb3be3c None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.081051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4869-d210-4761-8ef5-ed3d8fb3be3c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.081321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae4869-d210-4761-8ef5-ed3d8fb3be3c None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.083744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1244/2439] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:42.094637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.094925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.095204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.138587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:42.140211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:42.149143 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1196/2440] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:42.154754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.234782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e3d51-590a-48af-9f07-eceb42423e10 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.241078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:42.241854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.242279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.242752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.243671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:42.245760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:42.258447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:42.262595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1245/2441] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:51:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 24 20:51:42.276956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.278866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:42.293262 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1197/2442] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:42 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:42.297275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.350486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:42.429649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af27443-7e58-48b6-9b83-6235ddcaf0a5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.487385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49882dfd-8242-4c69-bc92-0c5b4f56dd4a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.494288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:42.495202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.495573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.496113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.497234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:42.515721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:42.547503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:42.550894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.551198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.551595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.562308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1198/2443] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:42 2022] POST /identity/v3/users => generated 298 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:42.566463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.590263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342011e-22b9-4432-8451-eec30e8cd6da admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.607022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:42.607793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.608174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.608546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.609322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:42.613733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:42.654123 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1199/2444] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:42.656486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.688210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1591-f4d4-487b-bea8-259e42955214 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.711662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:42.714136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.715267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.716551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] PATH_INFO: `/v3/projects/20538b579f4a4ca7bde2881eec23cb15/users/832f43f4c3a24185b4917c6bfaccc6ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.745036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:42.752075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:42.775524 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1246/2445] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:42.792204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20538b579f4a4ca7bde2881eec23cb15, user_id=832f43f4c3a24185b4917c6bfaccc6ad, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:42.795230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9cf7a-764a-4e3d-93e2-33b79ac420a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:42.834407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1200/2446] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:42 2022] PUT /identity/v3/projects/20538b579f4a4ca7bde2881eec23cb15/users/832f43f4c3a24185b4917c6bfaccc6ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:42.838422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:42.857021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea98f4b-f0d2-4099-bfbd-5978de5218f9 service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:42.920243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560fc79-6dc2-4417-a96c-3b9e9c6e7c30 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.921124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560fc79-6dc2-4417-a96c-3b9e9c6e7c30 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.921736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560fc79-6dc2-4417-a96c-3b9e9c6e7c30 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:42.923212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1201/2447] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:42.928531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e74f7a-bd8a-4a9d-b770-02aa4368700a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:42.928947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e74f7a-bd8a-4a9d-b770-02aa4368700a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:42.929316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e74f7a-bd8a-4a9d-b770-02aa4368700a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.009129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74e74f7a-bd8a-4a9d-b770-02aa4368700a None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:43.009925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74e74f7a-bd8a-4a9d-b770-02aa4368700a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:43.046806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.047427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.047703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.054081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.056668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.063113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.085558 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1247/2448] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:43.093126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:43.122847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62f6cb-6d30-43a6-8384-16e8675a55ce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:43.135479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.136205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.136527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.136811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] PATH_INFO: `/v3/projects/20538b579f4a4ca7bde2881eec23cb15/users/832f43f4c3a24185b4917c6bfaccc6ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.151509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20538b579f4a4ca7bde2881eec23cb15, user_id=832f43f4c3a24185b4917c6bfaccc6ad, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.154460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.165279 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1202/2449] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:51:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:43.168861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1248/2450] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:43 2022] PUT /identity/v3/projects/20538b579f4a4ca7bde2881eec23cb15/users/832f43f4c3a24185b4917c6bfaccc6ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:43.180705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac056d-b8f0-4914-a768-ee261dd8b2dc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.181771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac056d-b8f0-4914-a768-ee261dd8b2dc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.181771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac056d-b8f0-4914-a768-ee261dd8b2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.218114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:43.221907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:43.238253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:43.240617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6eac056d-b8f0-4914-a768-ee261dd8b2dc None None] MFA Rules not processed for user `832f43f4c3a24185b4917c6bfaccc6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:43.241998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6eac056d-b8f0-4914-a768-ee261dd8b2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:43.302350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:43.328676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40010b-c396-4d82-a861-fbad92869e90 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:43.332594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.333382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.333836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.334210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.345927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.348304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.363529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1249/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:43.363751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1203/2452] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:43.376586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:43.378208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:43.475487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76494936-caf5-483a-910c-c6df7810a12b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:43.482704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.483468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.483846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.484684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.495543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.497520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.508498 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1250/2453] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:43.546477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd82e0-f520-4b63-91ab-456c91b4bf72 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.546790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd82e0-f520-4b63-91ab-456c91b4bf72 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.547072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd82e0-f520-4b63-91ab-456c91b4bf72 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.548895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1204/2454] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:43.560760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:43.587787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:43.597550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7cc6-9940-4b34-a4ad-999da3d23ee0 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:43.602560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.603331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.603613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.603885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.605138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.607985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.609863 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1251/2455] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:43 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:43.699648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647c945-d014-4dd8-80ee-ff8eba845504 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.699999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647c945-d014-4dd8-80ee-ff8eba845504 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.700205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647c945-d014-4dd8-80ee-ff8eba845504 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.703644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1205/2456] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:43.734436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:43.767800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:43.797866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a4334-e21d-49f7-9104-6c4e7e7d7e2e tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:43.808860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:43.809887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:43.810381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:43.813130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:43.816308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:43.824095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:43.832226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1252/2457] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:43 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:44.087806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ba824-a9a2-4374-9180-b8e9420751fc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:44.088097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ba824-a9a2-4374-9180-b8e9420751fc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:44.088375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ba824-a9a2-4374-9180-b8e9420751fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:44.127759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0a0ba824-a9a2-4374-9180-b8e9420751fc None None] MFA Rules not processed for user `8c3a535cc471436689e916bd1cbc3159`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:44.128990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a0ba824-a9a2-4374-9180-b8e9420751fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:44.237671 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1206/2458] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:44.603761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:44.637933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37bad0-549c-4d11-b0a3-b5aecee08c35 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:44.658465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:44.661198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:44.663591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:44.669367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:44.700383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:44.702024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f31ee-675a-4a06-9f68-d17db75b237f service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:44.740936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1253/2459] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:45.039054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:45.039452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:45.039640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:45.083612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] MFA Rules not processed for user `6db52cb259fa426d860f9c9a48a149f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:45.088870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:45.261558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1207/2460] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:45.272956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c416-e22e-49c9-92ad-1be9c5f0cba6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:45.273242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c416-e22e-49c9-92ad-1be9c5f0cba6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:45.273529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1c416-e22e-49c9-92ad-1be9c5f0cba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:45.304684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-33a1c416-e22e-49c9-92ad-1be9c5f0cba6 None None] MFA Rules not processed for user `6db52cb259fa426d860f9c9a48a149f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:45.306098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33a1c416-e22e-49c9-92ad-1be9c5f0cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:45.324611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1254/2461] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:45.510931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:45.512821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:45.543262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9fe01-3f29-4434-8e11-dbe1af5fce46 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:45.549290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:45.550142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:45.550724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:45.551168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:45.560577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:45.562620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:45.574051 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1208/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:45 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:51:46.832351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5089a56-9f8f-44cd-b805-787314ea1fc6 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:46.832840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5089a56-9f8f-44cd-b805-787314ea1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:46.840553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5089a56-9f8f-44cd-b805-787314ea1fc6 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:46.842552 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1255/2463] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:46.847675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:46.862179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:46.887248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b788d0-5087-453f-92c0-2c7072bf9a04 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:46.894131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:46.895062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:46.895423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:46.895698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:46.897614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:46.899931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:46.902364 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1209/2464] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:46 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:47.152250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86dd3f-6cdb-4ed3-9b8a-560f2e3fb769 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.152530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86dd3f-6cdb-4ed3-9b8a-560f2e3fb769 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.152797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86dd3f-6cdb-4ed3-9b8a-560f2e3fb769 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.154117 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1256/2465] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:47.170137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:47.192336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:47.208842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ee123-5099-485d-8d18-25c8fe452fd6 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:47.214590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:47.215311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.215731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.216017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.217420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:47.219928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:47.222435 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1210/2466] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:47 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:47.308902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b815-995f-436e-874d-91b7f6b5b561 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.309492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b815-995f-436e-874d-91b7f6b5b561 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.310049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581b815-995f-436e-874d-91b7f6b5b561 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.312806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1257/2467] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:47.323202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:47.363376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:47.420930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f95e22-9ee0-4cca-945d-be8985a50bb7 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:47.433626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:47.435192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.436163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.437012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.440270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:47.446734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9201da-6d01-4e8d-80ad-ef06280f9060 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:47.455083 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1211/2468] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:47 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:47.472363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.472819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.473166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.537821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:47.539099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:47.692114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021dd15-1486-40cb-babe-faff3ce714d7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:47.692400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021dd15-1486-40cb-babe-faff3ce714d7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:47.692701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021dd15-1486-40cb-babe-faff3ce714d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:47.758029 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1258/2469] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:47.762331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4021dd15-1486-40cb-babe-faff3ce714d7 None None] MFA Rules not processed for user `8c3a535cc471436689e916bd1cbc3159`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:47.766210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4021dd15-1486-40cb-babe-faff3ce714d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:47.776882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1212/2470] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:47.784135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:47.789210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:47.802649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:48.019377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e2721-8d98-4281-9927-2e7b8dfff40c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:48.036882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:48.037828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:48.039163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:48.040279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:48.067502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:48.071909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:48.094217 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1259/2471] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:51:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:51:49.250485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e7209-97ef-4eb9-a8ce-135cd4f5b511 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.251094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e7209-97ef-4eb9-a8ce-135cd4f5b511 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.251094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e7209-97ef-4eb9-a8ce-135cd4f5b511 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.252681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1213/2472] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:49.259121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:49.282701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:49.319822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f728-eab4-47ad-b8cd-68392ca083ca service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:49.340527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:49.341505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.341997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.342482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.344524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:49.346849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:49.361305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1260/2473] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:49 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:49.711116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec79b7-0ea5-479c-93ac-bb7c36b9fed1 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.712051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec79b7-0ea5-479c-93ac-bb7c36b9fed1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.713093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec79b7-0ea5-479c-93ac-bb7c36b9fed1 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.714920 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1214/2474] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:49.735014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:49.758569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:49.783466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467528f3-6cb7-470a-8ca9-b53115567a8f tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:49.815884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:49.815884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.815884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.816288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.817267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:49.819722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:49.822928 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1261/2475] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:49 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:49.909467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e891-972c-40d4-b048-3c40e167db6d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.909749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e891-972c-40d4-b048-3c40e167db6d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.910021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e891-972c-40d4-b048-3c40e167db6d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.911992 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1215/2476] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:49.923247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:49.947907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:49.972346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ff8a6-b76e-43fd-9f14-b02942eada88 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:49.980715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:49.982130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:49.982626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:49.983304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:49.985201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:49.988028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7deb01-3785-4871-87e4-549ff43d6150 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:50.007135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1262/2477] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:49 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:50.165434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:50.165631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:50.166091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:50.204575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] MFA Rules not processed for user `8c3a535cc471436689e916bd1cbc3159`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:50.209140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:50.229117 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1216/2478] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:51.235108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec096ba-d8d1-408c-82b6-f558ef5b66ce None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:51.235691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec096ba-d8d1-408c-82b6-f558ef5b66ce None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:51.235815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec096ba-d8d1-408c-82b6-f558ef5b66ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:51.357431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0ec096ba-d8d1-408c-82b6-f558ef5b66ce None None] MFA Rules not processed for user `ca78cf6d34e84f2cb5cb254c5d8af1da`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:51.358776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0ec096ba-d8d1-408c-82b6-f558ef5b66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:51.613448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1263/2479] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:51.617116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:51.619259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:51.674843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:51.856803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f093a7-36eb-42c0-8f4c-5dd40da3e4a1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:51.882170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:51.887016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:51.887308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:51.887863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:51.888604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:51.893310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:51.951810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:51.952500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:51.953040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:51.955934 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1264/2480] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:51:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:51:51.968476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:51:51.984033 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1217/2481] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:51 2022] POST /identity/v3/projects => generated 397 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:51.987070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.006538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.019284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bea37-4246-4aae-956e-0bf629194f5a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.087830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:52.114171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '832f43f4c3a24185b4917c6bfaccc6ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20538b579f4a4ca7bde2881eec23cb15', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.114846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.115172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.115648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.118553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorizing `identity:get_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.121714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.139338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d17f1-3219-4b7c-a977-f6894289a5e7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.143137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1265/2482] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:51:51 2022] GET /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 381 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:51:52.154998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.155538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.155815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.156081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.156708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.161826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.208424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1218/2483] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:51 2022] POST /identity/v3/users => generated 305 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:52.213006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.230611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf70e-57b1-4fbd-ad45-d86b41bf3f00 tempest-QuotasAdminNegativeTest-1004569023 tempest-QuotasAdminNegativeTest-1004569023-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.245256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.245882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.246127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.246393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.247360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.249120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.280964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1266/2484] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:52.283263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.297895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.301609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b5684-9fd7-4e23-aa1f-6fe567f4ede6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.312891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614dc03-86f4-45c0-acbf-ad5f1a8804a0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.316216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.318935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.319230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.319505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] PATH_INFO: `/v3/projects/d53792b48f1248ec95d6cd73c056bb8e/users/238df9aa958f4ed8944c7d4355cb40c0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.324192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.324822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.325103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.325375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] PATH_INFO: `/v3/users/8c3a535cc471436689e916bd1cbc3159` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.327240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3a535cc471436689e916bd1cbc3159)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.330680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.372640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53792b48f1248ec95d6cd73c056bb8e, user_id=238df9aa958f4ed8944c7d4355cb40c0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.378527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.403832 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1219/2485] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:52 2022] PUT /identity/v3/projects/d53792b48f1248ec95d6cd73c056bb8e/users/238df9aa958f4ed8944c7d4355cb40c0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:52.414813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.435244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3a535cc471436689e916bd1cbc3159'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:52.437909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3a535cc471436689e916bd1cbc3159'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:52.452621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3a535cc471436689e916bd1cbc3159'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:52.484173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3a535cc471436689e916bd1cbc3159'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:52.484966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90895d54-f980-48a4-bcaf-3b834aeac7e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3a535cc471436689e916bd1cbc3159'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:52.517115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1267/2486] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:52 2022] DELETE /identity/v3/users/8c3a535cc471436689e916bd1cbc3159 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:51:52.521560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.521846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.522382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.558801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d56d86-3379-46f8-8be6-4ee7f2e3dde8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.580996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:52.604924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:52.609102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:52.776661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.778409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.781757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.782531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.784108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.786899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.802818 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1268/2487] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:52.828312 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1220/2488] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:52.832732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.834783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:52.863958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f290380-8bf6-4d56-8545-e645ad95b148 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:52.865779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:52.867662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:52.891829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:52.892553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:52.892834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:52.893142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] PATH_INFO: `/v3/projects/d53792b48f1248ec95d6cd73c056bb8e/users/238df9aa958f4ed8944c7d4355cb40c0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:52.906649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53792b48f1248ec95d6cd73c056bb8e, user_id=238df9aa958f4ed8944c7d4355cb40c0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:52.915858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:52.966085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1269/2489] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:52 2022] PUT /identity/v3/projects/d53792b48f1248ec95d6cd73c056bb8e/users/238df9aa958f4ed8944c7d4355cb40c0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:53.053145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13cf69-41be-44ac-9a7c-3aaeb3ae3e0c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:53.179703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:53.183726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:53.184035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:53.184282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:53.353919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:53.358713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:53.390788 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1221/2490] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:51:54.247525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:54.250761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:54.292747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46792e-e2d0-4a61-919f-97d0ca32d5fb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:54.302596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:54.303254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:54.303533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:54.303806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] PATH_INFO: `/v3/projects/e43a52b503c946659d54675548a49bab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:54.305550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43a52b503c946659d54675548a49bab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:54.308175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:54.412887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:54.459990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43a52b503c946659d54675548a49bab', 'request_id': 'req-f39e7a76-24a0-484a-9323-88fe7d763dbe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.472461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43a52b503c946659d54675548a49bab', 'request_id': 'req-f39e7a76-24a0-484a-9323-88fe7d763dbe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.517786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43a52b503c946659d54675548a49bab', 'request_id': 'req-f39e7a76-24a0-484a-9323-88fe7d763dbe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.540981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] The token cache is being invalidate because project e43a52b503c946659d54675548a49bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:54.542046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.543186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f39e7a76-24a0-484a-9323-88fe7d763dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.545415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1270/2491] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:54 2022] DELETE /identity/v3/projects/e43a52b503c946659d54675548a49bab => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:54.549487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:54.573120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9ea2-395b-4553-93ef-f146b04bac6f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:54.610080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:54.610540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:54.611165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:54.624746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:54.672641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] MFA Rules not processed for user `d278977b201c48dda2d73823fbfe4833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:54.679649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:54.828574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:54.829867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:54.830163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:54.835068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] PATH_INFO: `/v3/users/832f43f4c3a24185b4917c6bfaccc6ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:54.842909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] RBAC: Authorizing `identity:delete_user(user_id=832f43f4c3a24185b4917c6bfaccc6ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:54.856327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:54.893882 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1271/2492] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:51:54.979090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832f43f4c3a24185b4917c6bfaccc6ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.979867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832f43f4c3a24185b4917c6bfaccc6ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:54.997544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832f43f4c3a24185b4917c6bfaccc6ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:55.012662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832f43f4c3a24185b4917c6bfaccc6ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:55.013644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d971d83-c666-4597-98e4-955964fc348f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832f43f4c3a24185b4917c6bfaccc6ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:55.060040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1222/2493] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:51:54 2022] DELETE /identity/v3/users/832f43f4c3a24185b4917c6bfaccc6ad => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:55.830421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:55.832865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:55.859100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfb30f-eb21-4c15-b106-26c9d0c82c4b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:56.025240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:56.026103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.026363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.026719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] PATH_INFO: `/v3/projects/20538b579f4a4ca7bde2881eec23cb15` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.028111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20538b579f4a4ca7bde2881eec23cb15)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:56.030282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:56.116234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20538b579f4a4ca7bde2881eec23cb15', 'request_id': 'req-d907443e-c8fc-4190-aef0-adab7a02f544'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:56.126248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20538b579f4a4ca7bde2881eec23cb15', 'request_id': 'req-d907443e-c8fc-4190-aef0-adab7a02f544'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:56.143622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20538b579f4a4ca7bde2881eec23cb15', 'request_id': 'req-d907443e-c8fc-4190-aef0-adab7a02f544'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:56.164312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:56.173167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] The token cache is being invalidate because project 20538b579f4a4ca7bde2881eec23cb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:51:56.175711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:56.178430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d907443e-c8fc-4190-aef0-adab7a02f544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:51:56.187381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1272/2494] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:51:55 2022] DELETE /identity/v3/projects/20538b579f4a4ca7bde2881eec23cb15 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:56.188322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.188939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.188939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.239579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:56.240840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:56.512339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1223/2495] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:56.521676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.522273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.522590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.585365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:56.590078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:56.626605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:56.627256 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1273/2496] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:56.645433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ca07-afbf-49e8-bb67-6e7155abdf53 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:56.657276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:56.657911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.658189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.658462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.659163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:51:56.659568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:56.661203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:56.678794 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1224/2497] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:51:56.684367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:56.697618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15debcac-25c1-4176-85b4-bb0a7089e2ab None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:56.705114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:56.706455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.707079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.707513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.708302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:56.710380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:56.771132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1274/2498] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:56 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:51:56.777107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:56.792523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93475-ec26-4376-8ed0-4d14c9b113ef admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:56.839479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:56.950579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:56.951208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:56.951489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:56.951759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:56.956665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:56.963814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:57.016974 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1225/2499] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:56 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:57.021669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:57.037462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b907f-ec5a-4677-ba14-d84809af4fc6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:57.049314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:57.050106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.050392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.051285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.055574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:57.057247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:57.085414 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1275/2500] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:57.090071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:57.106376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e86d0-1aba-4f71-bce8-3d3d664d7718 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:57.126442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:57.127119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.127403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.127683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.128543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:57.130573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc127-c340-46da-a8cd-3445c77e135d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:57.148246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1226/2501] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:57.153109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:57.169351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17f53f-602e-4c4a-b566-846db07f701d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:57.183407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:57.184210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.184598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.187190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] PATH_INFO: `/v3/projects/73bec7c48b07450f95a29a35e113153d/users/936372eff4da45e281fc5512632a12b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.236279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bec7c48b07450f95a29a35e113153d, user_id=936372eff4da45e281fc5512632a12b6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:57.244758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:57.271648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1276/2502] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:57 2022] PUT /identity/v3/projects/73bec7c48b07450f95a29a35e113153d/users/936372eff4da45e281fc5512632a12b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:57.288829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.290074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.290538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.380997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:57.384822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:57.491793 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1227/2503] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:57.496924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:57.686361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418034a-b256-4c66-b448-616b0d1d02a8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:57.700517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:57.701305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.701648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.701772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.702473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:57.704549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:57.759612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1277/2504] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:51:57 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:57.765228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:57.795575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac3375-6ab5-4429-9b89-7a55fc83a4af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:57.994733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:57.996022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:57.996219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:57.996892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:57.997802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:58.011051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:58.061684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1228/2505] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:51:57 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:51:58.065585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:58.078478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d377a1-0f65-42e3-8428-8e6b6b4c9973 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:58.094668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:58.095384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:58.095733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:58.095921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:58.096690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:58.099099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:58.127889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:58.135006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1278/2506] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:58.142146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4c202-9e51-477e-8796-b31f08236db5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:58.162881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:58.163571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:58.164704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:58.164704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] PATH_INFO: `/v3/projects/05eca738faf2402faa48e15a456b8f54/users/fb94fdc9903e477d9dfb06cb82c5e365/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:58.201310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05eca738faf2402faa48e15a456b8f54, user_id=fb94fdc9903e477d9dfb06cb82c5e365, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:58.203554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:58.224048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:58.230614 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1229/2507] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:58 2022] PUT /identity/v3/projects/05eca738faf2402faa48e15a456b8f54/users/fb94fdc9903e477d9dfb06cb82c5e365/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:58.234576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:58.253331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b8d6b-c675-4e2e-b093-528ea614461a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:58.456659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:58.457284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:58.457562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:58.457831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:58.458471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:58.464733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbfbed-e8e2-43a7-af3d-56d3f4fcefa9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:58.512914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:51:58.516549 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1279/2508] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:51:58.529116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f19cd-8c7a-4fa9-a2cb-60166e57144f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:51:58.549288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:51:58.550424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:58.551010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:58.551627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] PATH_INFO: `/v3/projects/05eca738faf2402faa48e15a456b8f54/users/fb94fdc9903e477d9dfb06cb82c5e365/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:58.553983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05eca738faf2402faa48e15a456b8f54, user_id=fb94fdc9903e477d9dfb06cb82c5e365, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:51:58.560649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3633d-38f7-4ee7-ae03-69fabee43545 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:51:58.579854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1230/2509] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:51:58 2022] PUT /identity/v3/projects/05eca738faf2402faa48e15a456b8f54/users/fb94fdc9903e477d9dfb06cb82c5e365/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:51:58.592196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:58.592834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:58.594249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:58.616652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:51:58.668817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] MFA Rules not processed for user `fb94fdc9903e477d9dfb06cb82c5e365`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:58.672180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:59.020694 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1280/2510] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:51:59.022400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01610-3a46-42af-8940-eff5a061ff6b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:51:59.022764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01610-3a46-42af-8940-eff5a061ff6b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:51:59.023371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e01610-3a46-42af-8940-eff5a061ff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:51:59.091304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e1e01610-3a46-42af-8940-eff5a061ff6b None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:51:59.093028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1e01610-3a46-42af-8940-eff5a061ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:51:59.290728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1231/2511] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:51:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:00.135765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:00.137937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:00.343861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0529e-dc64-4c37-a7e8-26b269cc4106 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:00.391812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:00.396089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:00.396089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:00.396089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:00.456505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:00.466087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dd76f-5e0a-4519-97ec-c62599c21b44 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:00.533043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1281/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:04.671178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df447c-dc90-4425-b191-643dd14df007 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:04.671689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df447c-dc90-4425-b191-643dd14df007 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:04.671779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df447c-dc90-4425-b191-643dd14df007 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:04.673446 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1232/2513] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:52:04.680864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22c8c-3df3-4a88-9b1f-5f0ea7e7ceac None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:04.681404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22c8c-3df3-4a88-9b1f-5f0ea7e7ceac None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:04.681507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd22c8c-3df3-4a88-9b1f-5f0ea7e7ceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:04.736303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4fd22c8c-3df3-4a88-9b1f-5f0ea7e7ceac None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:04.740786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4fd22c8c-3df3-4a88-9b1f-5f0ea7e7ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:04.942589 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1282/2514] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:04.967341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df447c-dc90-4425-b191-643dd14df007 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:04.969244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-58df447c-dc90-4425-b191-643dd14df007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:05.000989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df447c-dc90-4425-b191-643dd14df007 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:05.013769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:05.014392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:05.014697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:05.014969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:05.027226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:05.028678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:05.048291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1233/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:06.996039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:06.996496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:06.997006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:07.031567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:07.032993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:07.089887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:52:07.145634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1283/2516] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:52:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:07.163065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:07.165945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:07.361768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6681352-70bd-420a-80b2-d112e586c695 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:07.375228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:07.376635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:07.377125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:07.377656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:07.392892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:07.394955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebc36f-6bc9-44ed-9c78-454e65b1eba0 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:07.415759 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1234/2517] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:52:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:52:08.475676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:08.525445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b659c12-e5da-4d4e-b685-d5065f11aaf7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:08.545839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:08.546473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:08.546787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:08.547053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:08.551205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:08.699133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:08.701020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf96ec3-9653-4666-9d84-eedadff361d6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:08.714188 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1284/2518] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:10.253570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:10.254684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:10.255251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:10.293919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] MFA Rules not processed for user `238df9aa958f4ed8944c7d4355cb40c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:10.295429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:10.372989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:52:10.406459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1235/2519] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:10.419251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:10.420150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:10.420606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:10.466520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] MFA Rules not processed for user `ca78cf6d34e84f2cb5cb254c5d8af1da`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:10.470689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:10.633452 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1285/2520] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:11.390084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:11.393899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:11.418154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f3a1e-ba82-4f61-962e-67531fa04eac None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:11.431802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:11.433512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:11.434027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:11.434522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:11.437379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:11.455127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:11.457648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac142ee-0c66-48ac-8731-72ebbb91c500 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:11.471305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1236/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:20.173982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:20.189150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e68ad0-ae48-4048-9bcd-48e27313ec9d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:20.196134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:20.197405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:20.197942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:20.198403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:20.208823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:20.211369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:20.234834 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1286/2522] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:21.068002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:21.068482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:21.068607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:21.088924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:21.090173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:21.097547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1237/2523] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:52:29.250776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:29.263671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd0c4e-d91e-4632-a94d-e22621bba507 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:29.269287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:29.269924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:29.270198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:29.270463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:29.280064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:29.281661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a0c94-db82-4c48-b6e1-646fb911ab82 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:29.294232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1287/2524] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:29 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:30.054803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:30.056269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:30.168583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7ba98-35d2-46bf-9066-54ebcadb7d9f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:30.195874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:30.196632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:30.196978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:30.197318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:30.200851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:30.223008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:30.225159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad490-7373-490d-8c60-0319484e45ef service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:30.236988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1238/2525] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:35.370198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423262-9479-41cb-b283-9e640506f31d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:35.371106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423262-9479-41cb-b283-9e640506f31d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:35.371382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35423262-9479-41cb-b283-9e640506f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:35.391662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-35423262-9479-41cb-b283-9e640506f31d None None] MFA Rules not processed for user `9c03c67e0c4c4de7a186f28ac10d0b92`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:35.393115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-35423262-9479-41cb-b283-9e640506f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:35.445967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:52:35.470629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1288/2526] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:35.936388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8955e7-a906-4891-a2db-e9cb6cb8ad09 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:35.937320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8955e7-a906-4891-a2db-e9cb6cb8ad09 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:35.938342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8955e7-a906-4891-a2db-e9cb6cb8ad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:35.975330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0c8955e7-a906-4891-a2db-e9cb6cb8ad09 None None] MFA Rules not processed for user `6db52cb259fa426d860f9c9a48a149f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:35.976698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0c8955e7-a906-4891-a2db-e9cb6cb8ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:36.074554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1239/2527] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:43.186092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423262-9479-41cb-b283-9e640506f31d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:43.187750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-35423262-9479-41cb-b283-9e640506f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:43.205632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35423262-9479-41cb-b283-9e640506f31d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:43.212320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:43.213370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:43.213878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:43.214391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:43.216584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:43.230175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:43.232145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cd5ff-7118-4820-bdcd-6d1d43625cfd service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:43.246213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1289/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:52:44.071922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6cd4-371e-495d-aa51-c5a4841ebc9a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:44.072406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6cd4-371e-495d-aa51-c5a4841ebc9a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:44.072498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce6cd4-371e-495d-aa51-c5a4841ebc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:44.079584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=71 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:52:44.093238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b1ce6cd4-371e-495d-aa51-c5a4841ebc9a None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:44.094442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1ce6cd4-371e-495d-aa51-c5a4841ebc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:44.103001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1240/2529] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:47.595304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:47.596241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:47.596616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:47.618870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] MFA Rules not processed for user `93982876b1e24efb8ffa8c009532343a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:47.620358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:47.720457 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1290/2530] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:52:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:49.825518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1895d-4305-46ec-a455-ddd0302f2844 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:49.826044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1895d-4305-46ec-a455-ddd0302f2844 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:49.826044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1895d-4305-46ec-a455-ddd0302f2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:49.893065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4ec1895d-4305-46ec-a455-ddd0302f2844 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:52:49.894363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ec1895d-4305-46ec-a455-ddd0302f2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:52:49.920040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1241/2531] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:52:49.970977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:52:50.025080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bcc8d-96b2-42f1-92be-24f63dc146d9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:52:50.040962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:52:50.041595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:52:50.041873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:52:50.042799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:52:50.055096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:52:50.058039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3bc3-f1fc-4262-aea3-e298cf70847b service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:52:50.069104 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1291/2532] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:52:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:52:59.873220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:00.064266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=42 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:01.088961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f874a-1917-44f4-8749-0a6d6bd21375 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:01.088961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f874a-1917-44f4-8749-0a6d6bd21375 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:01.088961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f874a-1917-44f4-8749-0a6d6bd21375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:01.109348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d79f874a-1917-44f4-8749-0a6d6bd21375 None None] MFA Rules not processed for user `ca78cf6d34e84f2cb5cb254c5d8af1da`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:01.110686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d79f874a-1917-44f4-8749-0a6d6bd21375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:01.122705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1242/2533] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:11.105346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:11.417672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:11.418604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:11.419022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:11.420545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1292/2534] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 20:53:11.429752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41deabc-750e-4ef5-a5ae-1c4790a4480f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:11.430255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41deabc-750e-4ef5-a5ae-1c4790a4480f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:11.430553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41deabc-750e-4ef5-a5ae-1c4790a4480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:11.451834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f41deabc-750e-4ef5-a5ae-1c4790a4480f None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:11.453129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f41deabc-750e-4ef5-a5ae-1c4790a4480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:11.462817 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1243/2535] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:11.475228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:11.478248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:11.496250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3da2a5-891c-43fd-99b8-20c6cd870c68 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:11.500760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:11.501743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:11.502708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:11.503361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:11.530624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:11.530624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcb6e2-9433-43c7-9c4b-335ad2c6b16b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:11.532865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1293/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:53:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:53:12.473856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:12.474344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:12.474447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:12.497783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:12.499818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:12.509995 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1244/2537] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:15.892959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d99477-90b5-42e0-ad1c-be23b0416086 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:15.894339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d99477-90b5-42e0-ad1c-be23b0416086 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:15.894339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d99477-90b5-42e0-ad1c-be23b0416086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:15.916294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d9d99477-90b5-42e0-ad1c-be23b0416086 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:15.917500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d9d99477-90b5-42e0-ad1c-be23b0416086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:15.925148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1294/2538] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:53:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:15.937956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:15.951670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f7c9-2ad8-480a-9451-3e3aeb40360c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:15.958080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:15.958826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:15.959170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:15.959519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:15.968496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:15.970274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77687450-eb78-4531-9024-ca7cf2c4c233 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:15.979990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1245/2539] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:53:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:53:25.911492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:25.976837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=12 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:27.000875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:27.001458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:27.001458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:27.024621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] MFA Rules not processed for user `6db52cb259fa426d860f9c9a48a149f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:27.025899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:27.033673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1295/2540] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:34.804415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92fd-9cef-4ad6-a820-2111197d1bcd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:34.804888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92fd-9cef-4ad6-a820-2111197d1bcd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:34.805093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c92fd-9cef-4ad6-a820-2111197d1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:34.825783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-901c92fd-9cef-4ad6-a820-2111197d1bcd None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:34.827363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-901c92fd-9cef-4ad6-a820-2111197d1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:34.853629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:34.856487 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1246/2541] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:34.868041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ae36-80f8-4516-ad38-35cf0e843b65 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:34.874865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:34.875622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:34.875986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:34.876349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:34.886188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:34.887939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:34.899588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1296/2542] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:53:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:53:36.827592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45ed5-bab9-46e2-b679-917fb5f5a067 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:36.828130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45ed5-bab9-46e2-b679-917fb5f5a067 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:36.828130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45ed5-bab9-46e2-b679-917fb5f5a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:36.846787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-45e45ed5-bab9-46e2-b679-917fb5f5a067 None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:36.847977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-45e45ed5-bab9-46e2-b679-917fb5f5a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:36.855011 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1247/2543] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:39.792509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:39.793906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:39.858389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d04c36-7897-4931-8529-3a87f080edf7 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:39.863680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:39.864307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:39.864585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:39.864856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] PATH_INFO: `/v3/users/8df2631dab974889a95bed3f863c0219` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:39.865964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df2631dab974889a95bed3f863c0219)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:39.867952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:39.921361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df2631dab974889a95bed3f863c0219'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:39.922037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df2631dab974889a95bed3f863c0219'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:39.930018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df2631dab974889a95bed3f863c0219'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:39.945168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df2631dab974889a95bed3f863c0219'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:39.945912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df2631dab974889a95bed3f863c0219'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:39.960164 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1297/2544] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:53:39 2022] DELETE /identity/v3/users/8df2631dab974889a95bed3f863c0219 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:39.964818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:39.965289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:39.965391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:39.984982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:39.986702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:40.050990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1248/2545] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:40.061384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:40.062990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:40.128879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed5d9d-9bae-4c99-8bb6-011b27dbb5eb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:40.132724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:40.133505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:40.133793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:40.134067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:40.147189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:40.149315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:40.159917 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1298/2546] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:53:40.493201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:40.495081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:40.515269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db2481-7e01-4c7c-aa09-c6698139600d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:40.521302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:40.522803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:40.523188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:40.523565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] PATH_INFO: `/v3/projects/899574bf215744b29695dc6128d4c8dd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:40.524937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] RBAC: Authorizing `identity:delete_project(project_id=899574bf215744b29695dc6128d4c8dd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:40.535101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:40.602565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899574bf215744b29695dc6128d4c8dd', 'request_id': 'req-81cec80a-4915-48db-bc6c-0a3a79c7245c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.616908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899574bf215744b29695dc6128d4c8dd', 'request_id': 'req-81cec80a-4915-48db-bc6c-0a3a79c7245c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.628799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899574bf215744b29695dc6128d4c8dd', 'request_id': 'req-81cec80a-4915-48db-bc6c-0a3a79c7245c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.652253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] The token cache is being invalidate because project 899574bf215744b29695dc6128d4c8dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:53:40.652903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.653722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.655960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1249/2547] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:53:40 2022] DELETE /identity/v3/projects/899574bf215744b29695dc6128d4c8dd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:53:40.660123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:40.665654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:40.672772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99d55-8978-471d-b52b-44e266623532 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:40.741436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:40.742072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:40.742349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:40.742654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] PATH_INFO: `/v3/users/d781afdc483d42c1bdcfbaa1294a4b31` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:40.743986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d781afdc483d42c1bdcfbaa1294a4b31)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:40.745914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:40.799042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd781afdc483d42c1bdcfbaa1294a4b31'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.800230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd781afdc483d42c1bdcfbaa1294a4b31'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.808039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd781afdc483d42c1bdcfbaa1294a4b31'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.820527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd781afdc483d42c1bdcfbaa1294a4b31'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.821480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcca38e7-2c2c-4733-9708-f539fcf9569a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd781afdc483d42c1bdcfbaa1294a4b31'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:40.833948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1299/2548] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:53:40 2022] DELETE /identity/v3/users/d781afdc483d42c1bdcfbaa1294a4b31 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:53:41.126246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.128361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:41.142042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cec80a-4915-48db-bc6c-0a3a79c7245c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.211154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.212154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.212618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.213226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] PATH_INFO: `/v3/projects/93c5cd84b7c44b258f25643417650f60` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.215102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c5cd84b7c44b258f25643417650f60)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.217255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.235177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:41.283495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c5cd84b7c44b258f25643417650f60', 'request_id': 'req-41c028dd-a8a4-40d3-8159-82f36fff19ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:41.290140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c5cd84b7c44b258f25643417650f60', 'request_id': 'req-41c028dd-a8a4-40d3-8159-82f36fff19ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:41.298445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c5cd84b7c44b258f25643417650f60', 'request_id': 'req-41c028dd-a8a4-40d3-8159-82f36fff19ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:41.314400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] The token cache is being invalidate because project 93c5cd84b7c44b258f25643417650f60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:53:41.314792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:41.315532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41c028dd-a8a4-40d3-8159-82f36fff19ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:41.317030 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1250/2549] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:53:41 2022] DELETE /identity/v3/projects/93c5cd84b7c44b258f25643417650f60 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:41.322167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.322557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.322849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.351578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:41.352943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:41.421089 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1300/2550] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:41.434414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.434727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.435002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.468962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:41.470205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:41.479284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1251/2551] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:41.483519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.493667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073ee8-49c4-4ece-9d80-73e9222ba248 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.498057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.498679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.498952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.499238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.499847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:53:41.500144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.501716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.513766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1301/2552] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:53:41.518606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.528344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c85a24-d5af-4350-93b2-65e482aacf98 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.533009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.533624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.533900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.534172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.534832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.536727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.563552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1252/2553] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:53:41 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:53:41.568312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.578266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0e1a0-a88f-4af5-8c91-5a74a50de144 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.583147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:41.665717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.666426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.666739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.667009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.667637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.669552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.704899 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1302/2554] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:53:41 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:53:41.707442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.726090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e84b-c5cb-409e-b153-329a312d1039 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.742044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.743004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.743289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.743563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.744269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.745813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.780693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1253/2555] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:53:41.786182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.819625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebed81a-0c74-4a7a-a10d-097080b36bcb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.834443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.835084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.835361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.835629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.836270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.838054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744bf53-9327-4b11-bb4f-674dbeeb6cac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.855124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1303/2556] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:53:41.856729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:41.872138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cf8ca-5995-4235-a2b0-b4c1908b0c2f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:41.877321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:41.891249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.893443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.895114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] PATH_INFO: `/v3/projects/66a0417c2ae142c48e41a41a495ea988/users/2d241b5c9b654fa7bf97f3652186ea6f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:41.921664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a0417c2ae142c48e41a41a495ea988, user_id=2d241b5c9b654fa7bf97f3652186ea6f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:41.950195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88869697-e41c-4144-a1ba-44170cf7ff59 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:41.981439 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1254/2557] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:53:41 2022] PUT /identity/v3/projects/66a0417c2ae142c48e41a41a495ea988/users/2d241b5c9b654fa7bf97f3652186ea6f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:41.992590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:41.993139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:41.993622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:42.046926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] MFA Rules not processed for user `2d241b5c9b654fa7bf97f3652186ea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:42.049821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:42.209940 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1304/2558] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:42.219324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:42.219865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:42.220255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:42.242918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] MFA Rules not processed for user `2d241b5c9b654fa7bf97f3652186ea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:42.244180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:42.251467 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1255/2559] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:42.262009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:42.265422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:42.361005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712ef4-0617-495e-b003-058bee3c8c39 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:42.370907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:42.371540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:42.371819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:42.372095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:42.380667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:42.382252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800e604-3ee6-4b77-9b74-c953d739f7cd service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:42.393297 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1305/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:53:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:53:52.237382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:52.388736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:56.324693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:56.326807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:56.478539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea623a7-49ec-4f67-b35e-39911ad0eaa4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:56.490950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:56.491608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:56.491888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:56.492162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] PATH_INFO: `/v3/users/9eec3a30bcc641ca8ecee19471007b1d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:56.493391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eec3a30bcc641ca8ecee19471007b1d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:56.495440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:56.590567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eec3a30bcc641ca8ecee19471007b1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:56.592494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eec3a30bcc641ca8ecee19471007b1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:56.610223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eec3a30bcc641ca8ecee19471007b1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:56.638011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eec3a30bcc641ca8ecee19471007b1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:56.643912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eec3a30bcc641ca8ecee19471007b1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:56.680665 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1256/2561] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:53:56 2022] DELETE /identity/v3/users/9eec3a30bcc641ca8ecee19471007b1d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:56.684790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:56.686639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:56.687086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:56.711864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:56.712827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:56.798822 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1306/2562] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:56.832329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:56.834022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:56.938523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71871558-8215-48d3-876f-68db815ba386 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:56.938523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:56.938523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:56.938523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:56.938887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:56.952819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:56.952819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:56.963947 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1257/2563] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:53:57.386767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:57.387949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:57.417875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba0063-1499-4c10-bead-3b81e40bb5e0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:57.433558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:57.434210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:57.434554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:57.434772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] PATH_INFO: `/v3/projects/5e68660c954f47f9bad1f7bee17f9435` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:57.435915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e68660c954f47f9bad1f7bee17f9435)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:57.441160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:57.532224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e68660c954f47f9bad1f7bee17f9435', 'request_id': 'req-c7df7035-06cd-4b98-aaaf-6c7e66492b15'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.539103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e68660c954f47f9bad1f7bee17f9435', 'request_id': 'req-c7df7035-06cd-4b98-aaaf-6c7e66492b15'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.546239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e68660c954f47f9bad1f7bee17f9435', 'request_id': 'req-c7df7035-06cd-4b98-aaaf-6c7e66492b15'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.597462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] The token cache is being invalidate because project 5e68660c954f47f9bad1f7bee17f9435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:53:57.598218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.598830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.602029 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1307/2564] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:53:57 2022] DELETE /identity/v3/projects/5e68660c954f47f9bad1f7bee17f9435 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:57.606166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:57.659742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:57.726164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368575c3-3623-446c-9d22-67060b4b9021 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:57.825098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:57.825852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:57.826247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:57.826653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] PATH_INFO: `/v3/users/9d29ecd81aad409594eff4f3fb16edf2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:57.827962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d29ecd81aad409594eff4f3fb16edf2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:57.831191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:57.929166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d29ecd81aad409594eff4f3fb16edf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.930211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d29ecd81aad409594eff4f3fb16edf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.947519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d29ecd81aad409594eff4f3fb16edf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.984511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d29ecd81aad409594eff4f3fb16edf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:57.987859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d95943d0-e6ac-4d7d-82fe-6d91c1d5802e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d29ecd81aad409594eff4f3fb16edf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.073922 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1258/2565] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:53:57 2022] DELETE /identity/v3/users/9d29ecd81aad409594eff4f3fb16edf2 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:58.508454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:58.510471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:58.653780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df7035-06cd-4b98-aaaf-6c7e66492b15 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:58.719131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:58.765848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:58.766673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:58.767033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:58.767418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] PATH_INFO: `/v3/projects/483f8a432244480698773200e1d49501` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:58.769990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483f8a432244480698773200e1d49501)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:58.773281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:58.845166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483f8a432244480698773200e1d49501', 'request_id': 'req-eb0ad17a-23a9-4663-b799-a250ef554a67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.853020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483f8a432244480698773200e1d49501', 'request_id': 'req-eb0ad17a-23a9-4663-b799-a250ef554a67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.861282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483f8a432244480698773200e1d49501', 'request_id': 'req-eb0ad17a-23a9-4663-b799-a250ef554a67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.881345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] The token cache is being invalidate because project 483f8a432244480698773200e1d49501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:53:58.881345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.882209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0ad17a-23a9-4663-b799-a250ef554a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:53:58.889617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1308/2566] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:53:58 2022] DELETE /identity/v3/projects/483f8a432244480698773200e1d49501 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:58.891209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:58.891719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:58.891814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.054103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:59.056212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:59.119155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:53:59.188309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1259/2567] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:59.191088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.191540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.192002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.229517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:59.230801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:59.255642 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1309/2568] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:53:59.256533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.272119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec5f01-99fa-446f-9d66-b78c91f3799e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.277954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.278941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.279330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.279722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.280490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:53:59.280888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.282701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.316620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1260/2569] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:53:59.321074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.332653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d38f2-1c85-4a19-928a-827eabc19421 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.348878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.349417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.349753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.350032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.350721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.352697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.402235 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1310/2570] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:53:59 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:53:59.406762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.421477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c56bcb-f438-472e-b71b-786087a89cb8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.498628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.499634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.500470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.501006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.503369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.506581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.534447 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1261/2571] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:53:59 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:53:59.538639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.553964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6d08a-5f58-48cc-8803-97ee5c46c8b6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.565992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.568209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.569879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.570869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.573855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.578379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.609410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1311/2572] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:53:59.614212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.630889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44709618-edb8-403e-a778-d485c8957c4a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.645662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.647748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.648459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.648970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.649833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.657104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0586e7-ecb3-4fab-ac02-6806a3ab2ca5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.692837 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1262/2573] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:53:59.695885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:53:59.719046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e2c7b-7f1a-4786-9420-8983d32efed8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:53:59.735042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:53:59.735915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.736306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.736753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] PATH_INFO: `/v3/projects/4b55ef17d925408093f5407fa8015787/users/2844f27200534323910d80cd869a6601/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.771874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b55ef17d925408093f5407fa8015787, user_id=2844f27200534323910d80cd869a6601, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:53:59.778568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:53:59.812261 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1312/2574] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:53:59 2022] PUT /identity/v3/projects/4b55ef17d925408093f5407fa8015787/users/2844f27200534323910d80cd869a6601/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:53:59.814277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:53:59.814617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:53:59.815424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:53:59.863357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] MFA Rules not processed for user `2844f27200534323910d80cd869a6601`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:53:59.864863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:53:59.995885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1263/2575] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:53:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:00.000013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.056881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:00.113364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f5270-4d6f-49a2-979b-e1aac7f7c592 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.131666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.132312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.132597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.132871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.133545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.135721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.183545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1313/2576] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:53:59 2022] POST /identity/v3/projects => generated 393 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:00.185699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.207419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bc71-e2c9-4336-97ce-052e04edaeaa None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.265032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:00.345082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.346957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.347430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.347862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.348852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.353341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.400523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1264/2577] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:00 2022] POST /identity/v3/users => generated 303 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:00.404443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.418070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeeb41-4ca0-44fd-8db8-97721825c3d4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.427533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.428162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.428955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.429352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.430084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.432570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.445782 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1314/2578] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:00.449992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.462037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcccb68a-8047-4037-a697-5e88f3a63f9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.467474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.468112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.468457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.468758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] PATH_INFO: `/v3/projects/2464ee933b394763a0135453d1072a2d/users/0f596f58695644da8f20168715392c49/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.485900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2464ee933b394763a0135453d1072a2d, user_id=0f596f58695644da8f20168715392c49, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.488682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.503448 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1265/2579] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:00 2022] PUT /identity/v3/projects/2464ee933b394763a0135453d1072a2d/users/0f596f58695644da8f20168715392c49/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:00.507599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.519980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf03e9-1f99-4764-a346-b1cf0c974acc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.589491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.589986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.589986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.590467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.591500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.593401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c61cb-c116-4ccb-9a83-4b0a56d7c0ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.609998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:00.618809 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1315/2580] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:00.623174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06446bf-9c5a-49e3-8f3c-4cae2016318d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:00.640075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:00.640906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.641196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.641473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] PATH_INFO: `/v3/projects/2464ee933b394763a0135453d1072a2d/users/0f596f58695644da8f20168715392c49/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.647135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2464ee933b394763a0135453d1072a2d, user_id=0f596f58695644da8f20168715392c49, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:00.650688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdf996-e934-4cc6-a284-716597351279 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:00.670815 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1266/2581] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:00 2022] PUT /identity/v3/projects/2464ee933b394763a0135453d1072a2d/users/0f596f58695644da8f20168715392c49/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:00.681569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07ec6-03bb-4309-afaa-b8b5d6c36be3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.681859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07ec6-03bb-4309-afaa-b8b5d6c36be3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.682146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d07ec6-03bb-4309-afaa-b8b5d6c36be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.731084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f1d07ec6-03bb-4309-afaa-b8b5d6c36be3 None None] MFA Rules not processed for user `0f596f58695644da8f20168715392c49`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:00.732585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f1d07ec6-03bb-4309-afaa-b8b5d6c36be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:00.847086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1316/2582] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:00.851207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:00.851576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:00.851940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:00.881547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] MFA Rules not processed for user `2844f27200534323910d80cd869a6601`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:00.882949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:00.943293 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1267/2583] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:01.016449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46665-0562-41f8-aa80-16ae852c33c2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:01.016799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46665-0562-41f8-aa80-16ae852c33c2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:01.017145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46665-0562-41f8-aa80-16ae852c33c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:01.036854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bab46665-0562-41f8-aa80-16ae852c33c2 None None] MFA Rules not processed for user `ca78cf6d34e84f2cb5cb254c5d8af1da`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:01.038166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bab46665-0562-41f8-aa80-16ae852c33c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:01.085345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:01.110109 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1317/2584] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:01.137966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:01.139592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:01.275857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d0c54-840e-4696-8227-8a6dada5bbe3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:01.282182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:01.283028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:01.283465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:01.284466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:01.298334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:01.310762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:01.313043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed7c32-6c68-411d-987b-9c412fdbe3fd service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:01.324555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1268/2585] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:03.479202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd197e-c613-43d6-82f9-87fbe531e56d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:03.479783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd197e-c613-43d6-82f9-87fbe531e56d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:03.479913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd197e-c613-43d6-82f9-87fbe531e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:03.526599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bbfd197e-c613-43d6-82f9-87fbe531e56d None None] MFA Rules not processed for user `936372eff4da45e281fc5512632a12b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:03.527859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bbfd197e-c613-43d6-82f9-87fbe531e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:03.633043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1318/2586] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:05.928299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:05.928859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:05.929313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:05.931211 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1269/2587] 10.222.0.52 () {54 vars in 921 bytes} [Sat Sep 24 20:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:54:05.965724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:05.966292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:05.967319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:06.017029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:06.018552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:06.103175 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1319/2588] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 20:54:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:06.274603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:06.276390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:06.295570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99096c49-2381-44b6-b7c7-9f24f050040b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:06.302457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:06.303368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:06.303569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:06.303845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:06.313270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:06.314930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:06.326330 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1270/2589] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:06.342796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:06.344231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:06.441439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677fab-5a92-4c87-99d3-3417d3d75cdd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:06.447707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:06.448373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:06.448657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:06.448935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] PATH_INFO: `/v3/users/6db52cb259fa426d860f9c9a48a149f3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:06.450106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db52cb259fa426d860f9c9a48a149f3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:06.452102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:06.485038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:06.536332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db52cb259fa426d860f9c9a48a149f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.537703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db52cb259fa426d860f9c9a48a149f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.547901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db52cb259fa426d860f9c9a48a149f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.569975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db52cb259fa426d860f9c9a48a149f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.570966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-42c8a957-44f0-4ce5-9025-e26173a29003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db52cb259fa426d860f9c9a48a149f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.585058 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1320/2590] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:06 2022] DELETE /identity/v3/users/6db52cb259fa426d860f9c9a48a149f3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:06.792858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:06.824075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab67b1-d3ec-43d2-88d0-a608002649a8 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:06.887959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:06.888576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:06.888854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:06.889118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] PATH_INFO: `/v3/projects/e175cb4032ec4e6aa900141abbaa04c4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:06.890238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e175cb4032ec4e6aa900141abbaa04c4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:06.892163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:06.946157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e175cb4032ec4e6aa900141abbaa04c4', 'request_id': 'req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.970027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e175cb4032ec4e6aa900141abbaa04c4', 'request_id': 'req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.980355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e175cb4032ec4e6aa900141abbaa04c4', 'request_id': 'req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.994289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] The token cache is being invalidate because project e175cb4032ec4e6aa900141abbaa04c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:06.994885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.995686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-baaccc7b-cf2b-41da-a8b3-4bed6ff2f2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:06.997222 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1271/2591] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:06 2022] DELETE /identity/v3/projects/e175cb4032ec4e6aa900141abbaa04c4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:07.002306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.002831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.002945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.022704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:07.023946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:07.102707 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1321/2592] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:07.110175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.110561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.111089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.151135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:07.152659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:07.163718 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1272/2593] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:07.167601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.182387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4935e-11ea-403d-9601-f09315e09412 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.192311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.192934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.193220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.193493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.194294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:07.194678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.197112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.218628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1322/2594] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:54:07.223734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.233820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6c826-3e4a-403c-95c4-82c4be213c6c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.240672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.241809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.242108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.242366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.243049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.246421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.287098 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1273/2595] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:07 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:07.289589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.305163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd59f7-959a-4a9c-af51-02fed6752508 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.427518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.428150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.428432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.428705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.429353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.432043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.467727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1323/2596] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:07 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:07.468454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.481001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091c4d5-d934-46ce-b156-bdc0e45491dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.489667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.489667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.489667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.489667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.490172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.491517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.506384 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1274/2597] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:07.509667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.522590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e462f-5815-473f-bff5-34b19c393ed8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.529693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.529931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.530286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.530601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.531244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.532837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd44d3d-bf5d-44fb-a657-8143d3ef4199 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.545977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1324/2598] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:07.549795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.573526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018e3f8-d631-473d-858a-c1482077e68d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.584090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.585464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.585751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.586023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] PATH_INFO: `/v3/projects/176a40378e334575814671210471a92c/users/50812c49f681493bbe428d7376572b3d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.612043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=176a40378e334575814671210471a92c, user_id=50812c49f681493bbe428d7376572b3d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.615780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.642540 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1275/2599] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:07 2022] PUT /identity/v3/projects/176a40378e334575814671210471a92c/users/50812c49f681493bbe428d7376572b3d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:07.653145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.653769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.654045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.675598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:07.698885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] MFA Rules not processed for user `50812c49f681493bbe428d7376572b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:07.702669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:07.811086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1325/2600] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:07.823639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:07.828264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:07.854601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:07.931924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5dab4-7543-4a87-88fc-b3d474021d9a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:07.946140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:07.946969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:07.947501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:07.947630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:07.956820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:07.958525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f58c3-b5af-4166-9d32-81c419433e57 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:07.967752 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1276/2601] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:11.401856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:11.441940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81447b86-964f-4dde-94c6-53fcb37fa9fb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:11.442330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81447b86-964f-4dde-94c6-53fcb37fa9fb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:11.442419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81447b86-964f-4dde-94c6-53fcb37fa9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:11.475735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-81447b86-964f-4dde-94c6-53fcb37fa9fb None None] MFA Rules not processed for user `93982876b1e24efb8ffa8c009532343a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:11.478259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-81447b86-964f-4dde-94c6-53fcb37fa9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:11.539473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da44e5-dc1a-4f4f-919a-f5358816b6d2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:11.548248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:11.550083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:11.550083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:11.550434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] PATH_INFO: `/v3/users/50812c49f681493bbe428d7376572b3d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:11.552009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50812c49f681493bbe428d7376572b3d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:11.555494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:11.610617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1277/2602] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:11.662038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50812c49f681493bbe428d7376572b3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:11.663310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50812c49f681493bbe428d7376572b3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:11.675223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50812c49f681493bbe428d7376572b3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:11.708729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50812c49f681493bbe428d7376572b3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:11.710020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50812c49f681493bbe428d7376572b3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:11.748097 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1326/2603] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:11 2022] DELETE /identity/v3/users/50812c49f681493bbe428d7376572b3d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:11.749628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:11.750481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:11.750999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:11.792326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:11.797127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:11.953141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1278/2604] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:12.011199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:12.013791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:12.055969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:12.178960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec21b-c312-47de-aa1e-e29cdee1c394 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:12.187435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:12.188241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:12.188633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:12.189007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:12.200215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:12.201773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096d7ad-828a-408a-be0f-a9cda4030a0f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:12.219653 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1327/2605] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:13.080728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.082321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:13.094906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e3a5e-d4d3-4806-979b-d5c1d4dd33e7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.101450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.102712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.103209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.103686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] PATH_INFO: `/v3/projects/176a40378e334575814671210471a92c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.105137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176a40378e334575814671210471a92c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.107663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.160627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:13.184107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176a40378e334575814671210471a92c', 'request_id': 'req-5803604a-7c27-4fb9-ac50-1702e3079210'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:13.192457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176a40378e334575814671210471a92c', 'request_id': 'req-5803604a-7c27-4fb9-ac50-1702e3079210'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:13.204144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176a40378e334575814671210471a92c', 'request_id': 'req-5803604a-7c27-4fb9-ac50-1702e3079210'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:13.223919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] The token cache is being invalidate because project 176a40378e334575814671210471a92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:13.224221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:13.224967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5803604a-7c27-4fb9-ac50-1702e3079210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:13.229848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1279/2606] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:13 2022] DELETE /identity/v3/projects/176a40378e334575814671210471a92c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:13.231794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.232282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.232645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.260920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:13.262622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:13.371896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1328/2607] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:13.383113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.383817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.384384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.410404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:13.412066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:13.427808 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1280/2608] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:13.438270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.451914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8ed1-3d68-450c-81db-a04416120715 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.457104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.457717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.457995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.458265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.460141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:13.461057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.463645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.488094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.490812 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1329/2609] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:54:13.498965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7be63-ce0d-41b0-9bc8-ac23f58d5d10 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.522674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.523302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.523587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.523865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.524532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.526441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.579335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1281/2610] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:13 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:13.581995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.603171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5313d-c950-4520-857a-3d0a98368e82 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.750628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.751662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.753499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.756296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.757016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.760785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.805138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1330/2611] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:13 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:13.809274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.830125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e8808-865c-4033-8af0-cfc2c391642f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.843092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.843715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.844612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.844891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.845544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.847294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.871047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1282/2612] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:13.875029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.890658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530d1b0-380a-4a68-8fff-86611df5acfa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.902883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.903577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.903932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.904378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.905094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.910898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1595b-20d5-4b05-b220-6e466e849560 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:13.932327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1331/2613] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:13.936374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:13.956005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d5e48-ab26-455a-bd36-3b02eef38310 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:13.960961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:13.961934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:13.962219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:13.962511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] PATH_INFO: `/v3/projects/8dc5dc16f54e4d2991a2b456216459a4/users/dab2b6bf40594154a6340d5afd02da0c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:13.989030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc5dc16f54e4d2991a2b456216459a4, user_id=dab2b6bf40594154a6340d5afd02da0c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:13.992036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:14.018600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1283/2614] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:13 2022] PUT /identity/v3/projects/8dc5dc16f54e4d2991a2b456216459a4/users/dab2b6bf40594154a6340d5afd02da0c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:14.022973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:14.023266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:14.023546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:14.045606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:14.081698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:14.085018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:14.290537 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1332/2615] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:14.301920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:14.305245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:14.410909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb7383-aa50-47be-a73a-c2b2ad33a676 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:14.416087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:14.417097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:14.418099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:14.418651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:14.436678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:14.439827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:14.455985 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1284/2616] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:19.104567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:19.106611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:19.120090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2e3c4-f1ea-4c72-81f8-1d095f852577 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:19.126138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:19.128129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:19.128609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:19.129112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] PATH_INFO: `/v3/users/d278977b201c48dda2d73823fbfe4833` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:19.130575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d278977b201c48dda2d73823fbfe4833)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:19.132898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:19.232134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd278977b201c48dda2d73823fbfe4833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:19.234197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd278977b201c48dda2d73823fbfe4833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:19.259018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd278977b201c48dda2d73823fbfe4833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:19.279843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd278977b201c48dda2d73823fbfe4833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:19.282411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd278977b201c48dda2d73823fbfe4833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:19.299325 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1333/2617] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:19 2022] DELETE /identity/v3/users/d278977b201c48dda2d73823fbfe4833 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:20.885624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:20.900221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9694289-6968-4896-904b-74030fa8e719 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:21.067592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:21.070163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:21.070163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:21.070163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] PATH_INFO: `/v3/projects/478a31be22cc451b92af447ed8d81242` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:21.075408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478a31be22cc451b92af447ed8d81242)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:21.079511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:21.084411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:21.186720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478a31be22cc451b92af447ed8d81242', 'request_id': 'req-6af69a7f-7dcd-4921-8364-e945fcac6807'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.194848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478a31be22cc451b92af447ed8d81242', 'request_id': 'req-6af69a7f-7dcd-4921-8364-e945fcac6807'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.212754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478a31be22cc451b92af447ed8d81242', 'request_id': 'req-6af69a7f-7dcd-4921-8364-e945fcac6807'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.232766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] The token cache is being invalidate because project 478a31be22cc451b92af447ed8d81242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:21.235078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.236037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.241274 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1285/2618] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:20 2022] DELETE /identity/v3/projects/478a31be22cc451b92af447ed8d81242 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:21.245117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:21.246684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:21.261703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f68df3-4ce0-4e8a-8b6b-31b5e5afcd37 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:21.360005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:21.363715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:21.366950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:21.367430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] PATH_INFO: `/v3/users/0686ece5b3ea46269d1098999ba98330` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:21.368994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0686ece5b3ea46269d1098999ba98330)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:21.374064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:21.406441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:21.445797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0686ece5b3ea46269d1098999ba98330'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.446845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0686ece5b3ea46269d1098999ba98330'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.458020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0686ece5b3ea46269d1098999ba98330'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.477764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0686ece5b3ea46269d1098999ba98330'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.478799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3184fe76-526f-4f88-9a3d-82354a3ab271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0686ece5b3ea46269d1098999ba98330'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:21.498325 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1334/2619] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:21 2022] DELETE /identity/v3/users/0686ece5b3ea46269d1098999ba98330 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:22.175520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.177493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:22.192878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af69a7f-7dcd-4921-8364-e945fcac6807 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.291224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.291865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.292144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.292411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] PATH_INFO: `/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.293564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197a3a5ec6ca4368a1fd953d2dbfd47c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.295512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.358588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197a3a5ec6ca4368a1fd953d2dbfd47c', 'request_id': 'req-cca801f4-6f82-4044-829d-e540d38461b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:22.366553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197a3a5ec6ca4368a1fd953d2dbfd47c', 'request_id': 'req-cca801f4-6f82-4044-829d-e540d38461b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:22.378909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197a3a5ec6ca4368a1fd953d2dbfd47c', 'request_id': 'req-cca801f4-6f82-4044-829d-e540d38461b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:22.395101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] The token cache is being invalidate because project 197a3a5ec6ca4368a1fd953d2dbfd47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:22.395388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:22.396550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cca801f4-6f82-4044-829d-e540d38461b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:22.402783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1286/2620] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:22 2022] DELETE /identity/v3/projects/197a3a5ec6ca4368a1fd953d2dbfd47c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:22.404081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.404544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.404709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.428722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:22.429965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:22.510047 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1335/2621] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:22.517451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.517737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.518011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.548628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:22.550409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:22.559175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1287/2622] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:22.563450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.575113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9adb56-7e06-4cd3-addb-1691766cffa1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.580069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.580931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.581399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.581961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.582831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:22.583139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.584937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.598947 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1336/2623] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:54:22.603548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.615452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b7927-ce57-40d2-a135-780e43d3164b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.620515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.622585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.623118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.623455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.625057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.627180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.656199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:22.677740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1288/2624] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:22 2022] POST /identity/v3/projects => generated 403 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:22.682405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.694419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537e11f-9ad1-4456-bcba-758ff3af24c5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.783509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.784899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.785254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.785601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.787253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.789284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.829764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.836021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1337/2625] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:22 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:22.849575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819e275-56f6-4255-9f39-36b767f7a475 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.855522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.872540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.873509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.877269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.879150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.880969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.912896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1289/2626] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:22.917143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:22.942167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af731ae0-49f5-4f71-8bde-51211fc2e254 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:22.950417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:22.951188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:22.951787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:22.952073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:22.961310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:22.963921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76110135-c5ba-4185-90fd-bc0e0a8a6b0c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:22.990204 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1338/2627] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:22.991286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.009291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928ad88-ec2d-4501-a9c7-6edd0808ee01 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:23.020451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:23.021347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.021615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.021891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] PATH_INFO: `/v3/projects/e470661b12a6454c8641b87827b073a6/users/821fe105082c42b096e348a48966471e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.055790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e470661b12a6454c8641b87827b073a6, user_id=821fe105082c42b096e348a48966471e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:23.059189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:23.094463 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1290/2628] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:22 2022] PUT /identity/v3/projects/e470661b12a6454c8641b87827b073a6/users/821fe105082c42b096e348a48966471e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:23.105668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.105955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.106305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.120738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:23.180455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] MFA Rules not processed for user `821fe105082c42b096e348a48966471e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:23.181734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:23.330054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1339/2629] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:23.336363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.450586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83204-5319-4d9c-9e6c-22b4092a27df admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:23.458716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:23.459466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.460426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.460426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.461404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:23.463705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a03b8-76e5-4f58-9e9e-36c1d366100c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:23.550766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.551309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1291/2630] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:23 2022] POST /identity/v3/projects => generated 403 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:23.579098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc16a06-6ff0-489d-a9ea-02ebdec3c845 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:23.730943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:23.734441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.738928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.739775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.740090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:23.748459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:23.779156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.779731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.780184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.797446 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1340/2631] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:23 2022] POST /identity/v3/users => generated 308 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:23.801867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.821632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6c7fc-36d3-4142-b431-f58f3df651d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:23.830814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:23.835955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:23.836818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:23.837443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.838555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.838855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.839717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:23.842105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:23.874221 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1341/2632] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:23.875045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.897655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c33096-4211-425a-8b87-44625af6f52a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:23.904894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:23.905577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:23.905858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:23.906133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] PATH_INFO: `/v3/projects/c6badc448b704fe29a75c0c0e40de963/users/42eef758668a4696aaa1d2505f77cb87/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:23.933847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6badc448b704fe29a75c0c0e40de963, user_id=42eef758668a4696aaa1d2505f77cb87, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:23.936512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:23.960902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1342/2633] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:23 2022] PUT /identity/v3/projects/c6badc448b704fe29a75c0c0e40de963/users/42eef758668a4696aaa1d2505f77cb87/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:23.962285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:23.965308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1292/2634] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:24.010564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:24.012636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:24.063813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:24.084451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694ad0-4190-4ae3-9f7d-550a2a4e6120 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:24.086730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:24.092377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:24.094865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:24.095249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:24.095754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:24.097205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:24.099487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:24.122353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0d982-992b-4a78-ac50-99f12ff80703 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:24.134211 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1343/2635] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:24.135367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:24.158583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6c75-9bad-40a9-acfb-8e437784979c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:24.170487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:24.177707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:24.178482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:24.178903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] PATH_INFO: `/v3/projects/c6badc448b704fe29a75c0c0e40de963/users/42eef758668a4696aaa1d2505f77cb87/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:24.182733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6badc448b704fe29a75c0c0e40de963, user_id=42eef758668a4696aaa1d2505f77cb87, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:24.187301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7afa6-1451-453d-a0dc-8e78256c2ec6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:24.234981 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1344/2636] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:24 2022] PUT /identity/v3/projects/c6badc448b704fe29a75c0c0e40de963/users/42eef758668a4696aaa1d2505f77cb87/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:24.239219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891950e-7d37-4654-b7b1-93926c3ad500 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:24.240138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891950e-7d37-4654-b7b1-93926c3ad500 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:24.240138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891950e-7d37-4654-b7b1-93926c3ad500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:24.267787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:24.272927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:24.273446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:24.273611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:24.293218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0891950e-7d37-4654-b7b1-93926c3ad500 None None] MFA Rules not processed for user `42eef758668a4696aaa1d2505f77cb87`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:24.294626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0891950e-7d37-4654-b7b1-93926c3ad500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:24.390724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:24.393797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:24.412660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1293/2637] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:54:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:54:24.443271 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1345/2638] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:24.883154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:24.884487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:25.074680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0303e-6528-4014-9410-57cf3792829e service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:25.128865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:25.201192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:25.202281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:25.203193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:25.203779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:25.229963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:25.245411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:25.290975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1294/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:26.166028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:26.166608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:26.166608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:26.194761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:26.196595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:26.221339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09af8e-a56f-4fd1-92f9-2fbbe9de121a service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:26.236438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] MFA Rules not processed for user `dab2b6bf40594154a6340d5afd02da0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:26.237876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:26.247633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:26.248289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:26.248573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:26.248861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:26.358988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:26.360608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:26.387424 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1346/2640] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:26.389485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1295/2641] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:26 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:26.864249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:26.887999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d66e6-3f6c-4c88-a9dc-173e3ec9af43 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:26.898725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:26.898725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:26.898973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:26.899295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:26.918444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:26.920938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61d047-b9f4-4dd3-8017-9bf5c8aaed52 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:26.982473 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1347/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:28.997546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:29.186188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca808b7b-cdd6-44b8-8b47-b6a63fd31817 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:29.193221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:29.194161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:29.195181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:29.195656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] PATH_INFO: `/v3/users/821fe105082c42b096e348a48966471e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:29.197425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821fe105082c42b096e348a48966471e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:29.201519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:29.309856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821fe105082c42b096e348a48966471e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:29.310768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821fe105082c42b096e348a48966471e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:29.330978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821fe105082c42b096e348a48966471e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:29.332968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:29.347006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821fe105082c42b096e348a48966471e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:29.347917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821fe105082c42b096e348a48966471e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:29.388783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1296/2643] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:28 2022] DELETE /identity/v3/users/821fe105082c42b096e348a48966471e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:29.394008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d5a3-a247-46b5-b0e2-8d224edf5208 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:29.395349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d5a3-a247-46b5-b0e2-8d224edf5208 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:29.395783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453d5a3-a247-46b5-b0e2-8d224edf5208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:29.432936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c453d5a3-a247-46b5-b0e2-8d224edf5208 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:29.441387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c453d5a3-a247-46b5-b0e2-8d224edf5208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:29.525545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:29.625226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1348/2644] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:29.660496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:29.663085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:29.869825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:29.870779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:29.871047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:29.911241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] MFA Rules not processed for user `dab2b6bf40594154a6340d5afd02da0c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:29.918751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dab6-852e-4d6b-8d82-b7e3bf7575a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:29.919005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:29.922426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:29.925014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:29.925626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:29.926702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:29.957613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:29.959223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:29.992021 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1297/2645] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:30.076135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1349/2646] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:30.599221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:30.602423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:30.624436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812586a-e1e9-446c-acad-1308e97e0562 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:30.629987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:30.630861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:30.631335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:30.631771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] PATH_INFO: `/v3/projects/e470661b12a6454c8641b87827b073a6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:30.633142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e470661b12a6454c8641b87827b073a6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:30.635310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:30.718595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e470661b12a6454c8641b87827b073a6', 'request_id': 'req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:30.735189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e470661b12a6454c8641b87827b073a6', 'request_id': 'req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:30.754314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e470661b12a6454c8641b87827b073a6', 'request_id': 'req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:30.785484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] The token cache is being invalidate because project e470661b12a6454c8641b87827b073a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:30.785794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:30.788234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:30.797383 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1298/2647] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:30 2022] DELETE /identity/v3/projects/e470661b12a6454c8641b87827b073a6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:30.798460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:30.821814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446d43d-b28b-4a84-a717-b3b2ddf8491c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:30.929258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:30.929833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:30.930157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:30.930489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] PATH_INFO: `/v3/users/42eef758668a4696aaa1d2505f77cb87` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:30.932944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42eef758668a4696aaa1d2505f77cb87)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:30.936100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:31.029076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42eef758668a4696aaa1d2505f77cb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:31.029997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42eef758668a4696aaa1d2505f77cb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:31.045149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42eef758668a4696aaa1d2505f77cb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:31.067329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42eef758668a4696aaa1d2505f77cb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:31.071034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-345ad64d-354a-46e8-9a62-0d2d33ff779b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42eef758668a4696aaa1d2505f77cb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:31.082138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:31.105384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1350/2648] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:30 2022] DELETE /identity/v3/users/42eef758668a4696aaa1d2505f77cb87 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:32.082102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:32.084692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:32.100152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fbdb0-b43a-4c67-8c2e-8a075e47780c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:32.197479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:32.211287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:32.212311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:32.213778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:32.214273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] PATH_INFO: `/v3/projects/c6badc448b704fe29a75c0c0e40de963` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:32.216884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6badc448b704fe29a75c0c0e40de963)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:32.223338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:32.354980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6badc448b704fe29a75c0c0e40de963', 'request_id': 'req-741830f3-61aa-416f-aa33-1a516154334d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:32.371471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6badc448b704fe29a75c0c0e40de963', 'request_id': 'req-741830f3-61aa-416f-aa33-1a516154334d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:32.386207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6badc448b704fe29a75c0c0e40de963', 'request_id': 'req-741830f3-61aa-416f-aa33-1a516154334d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:32.399578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] The token cache is being invalidate because project c6badc448b704fe29a75c0c0e40de963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:32.399945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:32.400762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-741830f3-61aa-416f-aa33-1a516154334d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:32.402577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1299/2649] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:32 2022] DELETE /identity/v3/projects/c6badc448b704fe29a75c0c0e40de963 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:32.407252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:32.408696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:32.410291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:32.434773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:32.436399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:32.572610 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1351/2650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:32.580338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:32.580908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:32.581260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:32.609882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:32.611482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:32.627043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1300/2651] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:32.629453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:32.643305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee230b5-0d89-47ba-b152-b1f0b5cac90d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:32.660676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:32.660676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:32.660676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:32.660676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:32.661794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:32.661794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:32.661794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:32.688578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1352/2652] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:32.692198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:32.725482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63100847-75e4-45a5-8b6c-98c50aa5ca25 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:32.740903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:32.742077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:32.743250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:32.744797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:32.745555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:32.750821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:32.881898 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1301/2653] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:32 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:32.883876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:32.907157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a26e-f326-46bb-9820-fae496f50f04 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:33.027425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:33.028692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.028692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.028882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.029539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:33.031516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:33.065918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1353/2654] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:32 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:33.067846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:33.089651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0b5ed-8def-43c7-8b01-4b5ddf5550cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:33.106285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:33.107806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.110139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.110516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.111549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:33.113739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:33.160465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1302/2655] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:33.162540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:33.186015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067e3a-0180-417a-b5d2-69310ee30a0c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:33.193311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:33.194363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.195328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.195634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.196782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:33.198379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123cd15-9b49-4608-b4f2-8e8e90650a8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:33.224467 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1354/2656] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:33.228611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:33.246540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d21444-ac0f-4752-9c0c-b0adfbf551ad admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:33.278432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:33.279830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.280342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.280905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] PATH_INFO: `/v3/projects/25b6e07b06884466b0991ba5f2b2cdfa/users/b1ffa312f28e4c04bfadeba0f905aa00/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.343080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b6e07b06884466b0991ba5f2b2cdfa, user_id=b1ffa312f28e4c04bfadeba0f905aa00, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:33.358605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:33.389735 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1303/2657] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:33 2022] PUT /identity/v3/projects/25b6e07b06884466b0991ba5f2b2cdfa/users/b1ffa312f28e4c04bfadeba0f905aa00/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:33.400648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.401215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.401707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.459499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] MFA Rules not processed for user `b1ffa312f28e4c04bfadeba0f905aa00`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:33.464443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:33.551200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:33.579812 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1355/2658] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:33.584497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:33.744129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5f5f0-fe32-4b6d-ae4c-23570a08a505 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:33.759334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:33.759963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:33.760242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:33.760512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:33.761150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:33.769769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:33.862041 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1304/2659] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:33 2022] POST /identity/v3/projects => generated 375 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:33.866055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:33.887788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dae915-8629-4bff-bf64-80cd2b0b82e6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:34.039995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:34.041093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.041518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.041802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.042962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:34.045123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:34.087245 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1356/2660] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:33 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:34.091074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:34.102102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e8d2-bcaa-4d82-b28e-98e28955806a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:34.107595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:34.108202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.108483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.108756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.109396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:34.111428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:34.116041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:34.126555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1305/2661] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:34.130751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:34.143354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bfdffd-0de7-491f-ab68-2808688763a4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:34.154149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:34.155282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.155744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.156186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] PATH_INFO: `/v3/projects/a616ce15d7034df49e5c99d95433cc41/users/29337d09556a47b9933b5073276fef8d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.180672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a616ce15d7034df49e5c99d95433cc41, user_id=29337d09556a47b9933b5073276fef8d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:34.184179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:34.209503 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1357/2662] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:34 2022] PUT /identity/v3/projects/a616ce15d7034df49e5c99d95433cc41/users/29337d09556a47b9933b5073276fef8d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:34.213950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:34.231428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22a789-4af3-4d67-9f64-cb03ddec5a54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:34.384603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:34.387046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.387494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.388055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.389473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:34.397931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7df85-29c6-4bc4-8c24-091086d985ba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:34.427838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1306/2663] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:34.428576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:34.440676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54626c37-c6e7-4cd3-bd6b-752e244275e7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:34.462833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:34.463654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.463940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.464207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] PATH_INFO: `/v3/projects/a616ce15d7034df49e5c99d95433cc41/users/29337d09556a47b9933b5073276fef8d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.470253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a616ce15d7034df49e5c99d95433cc41, user_id=29337d09556a47b9933b5073276fef8d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:34.476632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:34.510224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1358/2664] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:34 2022] PUT /identity/v3/projects/a616ce15d7034df49e5c99d95433cc41/users/29337d09556a47b9933b5073276fef8d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:34.525313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:34.525313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:34.525313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:34.599849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] MFA Rules not processed for user `29337d09556a47b9933b5073276fef8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:34.606910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:34.795307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1307/2665] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:34.973387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:34.976363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:35.106470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75939b14-ca2f-4837-8cec-6964d5ca9ddb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:35.113182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:35.114471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:35.114999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:35.116239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:35.127146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:35.131237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec6ccd-1c39-4d92-b7d9-09e79d12a384 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:35.144235 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1359/2666] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:35.269465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:35.275724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:35.306854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef746d-b117-41a2-9608-04665108e6eb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:35.331179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:35.332081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:35.332457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:35.332870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:35.335856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:35.498422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:35.500667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:35.515192 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1308/2667] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:35 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:35.552277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:35.552836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:35.553061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:35.637805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] MFA Rules not processed for user `2844f27200534323910d80cd869a6601`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:35.639288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:35.690045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:35.705107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:35.725525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f353680-4e97-4354-b7c9-d51bd21f4443 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:35.747577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1360/2668] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:35.752033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:35.753276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:35.753566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:35.753867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:35.780151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:35.895655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:35.899922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600a7c-358a-4b66-ad47-de83087865cd service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:35.927257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1309/2669] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:39.345777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:39.451288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc83af-13ca-4ae0-bf4f-4a8014685a0f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:39.456597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:39.458070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:39.458803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:39.459277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] PATH_INFO: `/v3/users/b1ffa312f28e4c04bfadeba0f905aa00` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:39.462201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ffa312f28e4c04bfadeba0f905aa00)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:39.464678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:39.561439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ffa312f28e4c04bfadeba0f905aa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:39.562152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ffa312f28e4c04bfadeba0f905aa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:39.573497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ffa312f28e4c04bfadeba0f905aa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:39.594002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ffa312f28e4c04bfadeba0f905aa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:39.594790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ffa312f28e4c04bfadeba0f905aa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:39.614745 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1361/2670] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:39 2022] DELETE /identity/v3/users/b1ffa312f28e4c04bfadeba0f905aa00 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:39.619170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:39.619170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:39.619170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:39.641131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:39.642766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:39.791145 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1310/2671] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:39.799815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:39.801515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:39.896788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e35eb-d073-4640-a83b-49ebeaa3fbef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:39.900853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:39.901429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:39.901740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:39.902465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:39.912880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:39.917518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:39.921248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:39.938232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1362/2672] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:54:40.674112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:40.675644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:40.693103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a475ba-5818-4f47-a06e-d498a314cb1b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:40.698157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:40.698827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:40.699303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:40.699750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] PATH_INFO: `/v3/projects/25b6e07b06884466b0991ba5f2b2cdfa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:40.701067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b6e07b06884466b0991ba5f2b2cdfa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:40.703223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:40.801838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b6e07b06884466b0991ba5f2b2cdfa', 'request_id': 'req-c88cc3ca-7c85-469b-9562-1218fd19f7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:40.823998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b6e07b06884466b0991ba5f2b2cdfa', 'request_id': 'req-c88cc3ca-7c85-469b-9562-1218fd19f7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:40.834173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b6e07b06884466b0991ba5f2b2cdfa', 'request_id': 'req-c88cc3ca-7c85-469b-9562-1218fd19f7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:40.856100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] The token cache is being invalidate because project 25b6e07b06884466b0991ba5f2b2cdfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:40.856868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:40.858207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:40.861125 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1311/2673] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:40 2022] DELETE /identity/v3/projects/25b6e07b06884466b0991ba5f2b2cdfa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:40.865098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:40.878447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7259902-c63a-47fa-ba18-74f87ef1d6e9 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:40.994776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:40.995797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:40.996153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:40.996501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] PATH_INFO: `/v3/users/29337d09556a47b9933b5073276fef8d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:40.998721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29337d09556a47b9933b5073276fef8d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:41.001609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:41.135525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29337d09556a47b9933b5073276fef8d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:41.136763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29337d09556a47b9933b5073276fef8d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:41.155159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29337d09556a47b9933b5073276fef8d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:41.199188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29337d09556a47b9933b5073276fef8d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:41.199948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c68add7c-7c5b-4a42-b2bc-1112848d0dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29337d09556a47b9933b5073276fef8d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:41.235147 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1363/2674] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:40 2022] DELETE /identity/v3/users/29337d09556a47b9933b5073276fef8d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:42.597632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:42.600623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:42.625044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cc3ca-7c85-469b-9562-1218fd19f7bb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:42.716723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:42.778511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:42.779294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:42.779585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:42.779883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] PATH_INFO: `/v3/projects/a616ce15d7034df49e5c99d95433cc41` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:42.784677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a616ce15d7034df49e5c99d95433cc41)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:42.786979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:42.952861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a616ce15d7034df49e5c99d95433cc41', 'request_id': 'req-3513b330-062c-42c6-b4a9-5b6629aa20b1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:42.972659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a616ce15d7034df49e5c99d95433cc41', 'request_id': 'req-3513b330-062c-42c6-b4a9-5b6629aa20b1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:42.991167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a616ce15d7034df49e5c99d95433cc41', 'request_id': 'req-3513b330-062c-42c6-b4a9-5b6629aa20b1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:43.036097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] The token cache is being invalidate because project a616ce15d7034df49e5c99d95433cc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:43.041599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:43.049493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3513b330-062c-42c6-b4a9-5b6629aa20b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:43.054759 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1312/2675] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:42 2022] DELETE /identity/v3/projects/a616ce15d7034df49e5c99d95433cc41 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:43.059797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.060258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.060258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.106222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:43.110763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:43.292398 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1364/2676] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:43.303940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.304223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.304500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.351863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:43.356372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:43.405038 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1313/2677] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:43.411857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.424275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4809a95-561d-4d30-9882-6517d270d341 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:43.452033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:43.452738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.453022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.453765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.454134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:43.454430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:43.456357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:43.535960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.545910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9a5d-0f22-432c-8d5c-6b1bdd5b79ef None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:43.548918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1365/2678] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:43.560672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:43.561480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.561828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.562152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.562909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:43.564860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:43.605652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.611560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1314/2679] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:43 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:43.612158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:43.620864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8ef63-6e7f-4970-be77-5d8540a2e408 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:43.764620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:43.767368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.769295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.769938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.771384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:43.777511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:43.841922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.843132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1366/2680] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:43 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:43.857651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28613f2-7f19-4819-80ed-da2344c7c7a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:43.866050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:43.868146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.868662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.871683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.872868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:43.875567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:43.901930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1315/2681] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:43.905812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.921151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a6da-bf16-4a14-9c9e-43dd6964bd39 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:43.934361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:43.937418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:43.938636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:43.941301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:43.942475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:43.945315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a05298-2ada-4cb5-b85b-766cda9ede01 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:43.977073 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1367/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:43.980133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:43.994959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbf71f-88ac-445a-9068-c7bb5b0493d9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:44.007436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:44.009151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.010956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.011298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] PATH_INFO: `/v3/projects/0612be8c15d446e3856c0aa5830f5065/users/a734c7f617584b5ca4d8dad6219abc52/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.053560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0612be8c15d446e3856c0aa5830f5065, user_id=a734c7f617584b5ca4d8dad6219abc52, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:44.056789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:44.088900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1316/2683] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:43 2022] PUT /identity/v3/projects/0612be8c15d446e3856c0aa5830f5065/users/a734c7f617584b5ca4d8dad6219abc52/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:44.102634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.103790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.104287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.173054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] MFA Rules not processed for user `a734c7f617584b5ca4d8dad6219abc52`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:44.180787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:44.387415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1368/2684] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:44.391241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:44.564878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acd121-587f-441c-9cfd-0d8cacd37da4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:44.576625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:44.576885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.577697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.577902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.578566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:44.581228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:44.590391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:44.666758 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1317/2685] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:44 2022] POST /identity/v3/projects => generated 371 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:44.667324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:44.692977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f06534-438b-44ef-a6a1-f5db2a2f5941 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:44.837316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:44.837316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.837316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.837316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.838329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:44.838329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:44.858335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:44.882828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:44.883206 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1369/2686] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:44 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:44.896436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbd8e2-bdba-4bcb-ab8d-0e6e1c003fb3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:44.905217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:44.906061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.907165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.908261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.910677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:44.913364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:44.943180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:44.954280 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1318/2687] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:44.955793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789460dd-1b4b-44b8-a832-8bd5803e0d39 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:44.961235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:44.962177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:44.962714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:44.963377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] PATH_INFO: `/v3/projects/15158ac951e848028ad355cdd84b0e9f/users/e3b8ce994ec74e57bf27b3a3931fe27d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:44.985572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15158ac951e848028ad355cdd84b0e9f, user_id=e3b8ce994ec74e57bf27b3a3931fe27d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:44.992886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:45.023611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1370/2688] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:44 2022] PUT /identity/v3/projects/15158ac951e848028ad355cdd84b0e9f/users/e3b8ce994ec74e57bf27b3a3931fe27d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:45.041628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:45.081798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb3d1d-f0bb-4bdd-877a-9595a2734cbd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:45.199983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:45.200934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:45.200934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:45.201164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:45.201872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:45.203658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225e7f-94e6-4e6c-ab01-bb4c1afbb1d2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:45.222684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1319/2689] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:45.227719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:45.258067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82444-d8cc-4472-99cc-515f8d4fa73d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:45.263055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:45.263747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:45.264046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:45.264315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] PATH_INFO: `/v3/projects/15158ac951e848028ad355cdd84b0e9f/users/e3b8ce994ec74e57bf27b3a3931fe27d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:45.266256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] RBAC: Authorizing `identity:create_grant(project_id=15158ac951e848028ad355cdd84b0e9f, user_id=e3b8ce994ec74e57bf27b3a3931fe27d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:45.269883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:45.284152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1371/2690] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:45 2022] PUT /identity/v3/projects/15158ac951e848028ad355cdd84b0e9f/users/e3b8ce994ec74e57bf27b3a3931fe27d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:45.294928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:45.295230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:45.295501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:45.328602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] MFA Rules not processed for user `e3b8ce994ec74e57bf27b3a3931fe27d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:45.329448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:45.420281 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1320/2691] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:45.428373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:45.430941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:45.514736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696009b-ba13-4828-a4d0-3d2129ca79da admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:45.521322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:45.521927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:45.522209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:45.522482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:45.530982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:45.532556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b97ba-1e54-4e75-85e2-15a384c74307 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:45.542035 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1372/2692] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:46.048275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:46.153338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:46.161523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9b518-54cb-4ff5-8025-9788c6611053 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:46.169867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:46.170834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:46.171339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:46.171771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] PATH_INFO: `/v3/users/a734c7f617584b5ca4d8dad6219abc52` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:46.173163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a734c7f617584b5ca4d8dad6219abc52)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:46.175710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:46.249380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a734c7f617584b5ca4d8dad6219abc52'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:46.250532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a734c7f617584b5ca4d8dad6219abc52'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:46.261323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a734c7f617584b5ca4d8dad6219abc52'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:46.273025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a734c7f617584b5ca4d8dad6219abc52'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:46.274274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a734c7f617584b5ca4d8dad6219abc52'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:46.297118 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1321/2693] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:46 2022] DELETE /identity/v3/users/a734c7f617584b5ca4d8dad6219abc52 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:46.301891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:46.302863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:46.303299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:46.325107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:46.326366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:46.468178 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1373/2694] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:46.477512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:46.479688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:46.568059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733a25f-7b98-48ff-9917-1d55c29f4819 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:46.572276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:46.572893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:46.573167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:46.573434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:46.584318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:46.585891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:46.597793 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1322/2695] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:47.539415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:47.539415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:47.561290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790494d-ce9a-4e45-9989-3f30cdc7c69d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:47.576070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:47.577423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:47.577710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:47.578053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] PATH_INFO: `/v3/projects/0612be8c15d446e3856c0aa5830f5065` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:47.582740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0612be8c15d446e3856c0aa5830f5065)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:47.584681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:47.703781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:47.710728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0612be8c15d446e3856c0aa5830f5065', 'request_id': 'req-8a46ef22-0368-4c42-8804-f1db44ff3397'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.720857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0612be8c15d446e3856c0aa5830f5065', 'request_id': 'req-8a46ef22-0368-4c42-8804-f1db44ff3397'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.734646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0612be8c15d446e3856c0aa5830f5065', 'request_id': 'req-8a46ef22-0368-4c42-8804-f1db44ff3397'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.759125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] The token cache is being invalidate because project 0612be8c15d446e3856c0aa5830f5065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:47.759667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.760509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.762380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1374/2696] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:47 2022] DELETE /identity/v3/projects/0612be8c15d446e3856c0aa5830f5065 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:47.766698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:47.782319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6598059-8e8e-490e-85b9-ccc3a431d1be service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:47.911535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:47.912468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:47.912900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:47.913495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] PATH_INFO: `/v3/users/e3b8ce994ec74e57bf27b3a3931fe27d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:47.916281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b8ce994ec74e57bf27b3a3931fe27d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:47.919130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:47.952706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:47.992302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b8ce994ec74e57bf27b3a3931fe27d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:47.993427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b8ce994ec74e57bf27b3a3931fe27d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.005165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b8ce994ec74e57bf27b3a3931fe27d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.016716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b8ce994ec74e57bf27b3a3931fe27d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.017895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b8ce994ec74e57bf27b3a3931fe27d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.051996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1323/2697] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:47 2022] DELETE /identity/v3/users/e3b8ce994ec74e57bf27b3a3931fe27d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:48.477669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:48.479749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:48.496144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46ef22-0368-4c42-8804-f1db44ff3397 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:48.521411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:48.523154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:48.573834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:48.575503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.575952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.577002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] PATH_INFO: `/v3/projects/15158ac951e848028ad355cdd84b0e9f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.578316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15158ac951e848028ad355cdd84b0e9f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:48.580987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:48.593385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6cc2-eb87-4f60-aea6-828b910bdc54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:48.597744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:48.598881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.599316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.599936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.602254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:48.633016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15158ac951e848028ad355cdd84b0e9f', 'request_id': 'req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.639661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15158ac951e848028ad355cdd84b0e9f', 'request_id': 'req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.646833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15158ac951e848028ad355cdd84b0e9f', 'request_id': 'req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.664058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] The token cache is being invalidate because project 15158ac951e848028ad355cdd84b0e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:48.664355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.665278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ff78ec54-40cc-4e5f-a61d-9b494e50cbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:48.667599 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1375/2698] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:48 2022] DELETE /identity/v3/projects/15158ac951e848028ad355cdd84b0e9f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:48.675893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.676533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.676881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.687935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:48.689623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae73184-40fd-4bbe-a17e-5e86a98b0f68 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:48.699798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:48.701132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:48.704646 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1324/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:54:48.791196 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1376/2700] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:48.794559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.795024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.795143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.817438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:48.818959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:48.827613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1325/2701] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:48.832680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:48.837793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:48.843410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f54fc6-1be1-442c-ac23-bbc68d7ef7d6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:48.850659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:48.851481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.852007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.852484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.853252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:48.853639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:48.856510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:48.875430 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1377/2702] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:54:48.880111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:48.889544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66aa04-c26a-4cae-8f55-2b9b85d0c6fc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:48.894810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:48.895429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:48.895704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:48.895974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:48.896604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:48.898867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:48.928474 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1326/2703] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:48 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:48.932747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:48.943434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98377c07-7421-4c29-9f09-9ef95b9efcc6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.024111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.024744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.025028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.025824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.025959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.028051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.060388 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1378/2704] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:48 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:49.061789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.075361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf9871-66d3-4efd-b2f6-07f922e24bcd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.080796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.082006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.082286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.082607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.083257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.085536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28ab1a-5ef5-4650-bc7e-0f76e65fc302 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.091772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:49.103885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1327/2705] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:49.105910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.116660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396bb66-74a3-4411-9893-6cc1f0172a73 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.121395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.121988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.122262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.122565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.122806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.123098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.123350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.123460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.125022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.139519 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1379/2706] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:49.144488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.152458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:49.153907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:49.155621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070c517-fe21-4c57-805c-2dbcb20a22ef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.161275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.161942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.162226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.162527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] PATH_INFO: `/v3/projects/6d21d03d92e1426f8043940883de088c/users/a4c25b0e318745b7a64f0456847ee201/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.178716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d21d03d92e1426f8043940883de088c, user_id=a4c25b0e318745b7a64f0456847ee201, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.181409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e4609-5a08-4bce-bba5-af8774436357 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.196624 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1380/2707] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:49 2022] PUT /identity/v3/projects/6d21d03d92e1426f8043940883de088c/users/a4c25b0e318745b7a64f0456847ee201/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:49.208312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.208601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.208865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.219304 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1328/2708] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:49.233300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.235643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:49.242331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] MFA Rules not processed for user `a4c25b0e318745b7a64f0456847ee201`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:49.243953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:49.318247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ff8b5-730f-4bc8-9339-b9e2c45e30ab None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.320470 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1381/2709] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:49.326837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.366644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:49.394608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.396170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.396837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.396837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.408520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b490315-6c99-47d7-857b-e29b4a063430 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.416018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.417115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.417396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.417810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.418594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.420880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.459210 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1382/2710] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:49 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:49.462581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.473256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:49.486544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.491932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.509298 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1329/2711] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:54:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:54:49.541375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3ed8-f3ea-4e74-b7a2-f497bea233d5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.546366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.547165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.547454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.547726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.548404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.550872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.578545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1383/2712] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:49 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:49.587011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.607056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0beadf-eae5-4fa3-8e17-da01c3b417ec service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.613333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.613953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.614237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.614624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.615345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.617204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.631677 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1330/2713] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:49.641564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.660033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71a38f-fc58-4217-a86e-d2611f511e57 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.675873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.676654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.676996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.677336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] PATH_INFO: `/v3/projects/d802560be740486e946ded17fbf68906/users/4937765d76e44f1ca2f3390f4f2e5966/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.695726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d802560be740486e946ded17fbf68906, user_id=4937765d76e44f1ca2f3390f4f2e5966, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.700062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.715365 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1384/2714] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:49 2022] PUT /identity/v3/projects/d802560be740486e946ded17fbf68906/users/4937765d76e44f1ca2f3390f4f2e5966/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:49.719394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.732081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ad60-9822-4a42-b315-4dc72cd112ca admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.828222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.830290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.830857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.832360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.833233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.836059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e04cc-8ca5-4668-b6cd-dda5a1821e60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.865496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1331/2715] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:49.869689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:49.881047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fbeca-3989-4712-8471-a8011094f3eb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:49.890986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:49.891797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.892172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.892547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] PATH_INFO: `/v3/projects/d802560be740486e946ded17fbf68906/users/4937765d76e44f1ca2f3390f4f2e5966/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.904719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d802560be740486e946ded17fbf68906, user_id=4937765d76e44f1ca2f3390f4f2e5966, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:49.909451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:49.926785 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1385/2716] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:49 2022] PUT /identity/v3/projects/d802560be740486e946ded17fbf68906/users/4937765d76e44f1ca2f3390f4f2e5966/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:49.934927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:49.934927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:49.934927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:49.972831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] MFA Rules not processed for user `4937765d76e44f1ca2f3390f4f2e5966`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:49.974623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:50.093559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1332/2717] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:50.230414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:50.232283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:50.332319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f484b7-0a32-47c0-a2ee-082fe2a8fc95 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:50.338232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:50.339378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:50.340092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:50.340092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:50.348283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:50.352980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:50.354853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7fcbc-3969-477d-a437-b6371a0e620c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:50.367152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1386/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:50.468903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:50.535861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b46761-2e17-461c-ad31-24182e87f92e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:50.540440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:50.541064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:50.541341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:50.541612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] PATH_INFO: `/v3/users/a4c25b0e318745b7a64f0456847ee201` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:50.542830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c25b0e318745b7a64f0456847ee201)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:50.544762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:50.546358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:50.601217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c25b0e318745b7a64f0456847ee201'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:50.601916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c25b0e318745b7a64f0456847ee201'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:50.612945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c25b0e318745b7a64f0456847ee201'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:50.622839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c25b0e318745b7a64f0456847ee201'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:50.623892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c25b0e318745b7a64f0456847ee201'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:50.653613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1333/2719] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:50 2022] DELETE /identity/v3/users/a4c25b0e318745b7a64f0456847ee201 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:50.657755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:50.658210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:50.658311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:50.695629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:50.696873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:50.794348 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1387/2720] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:50.807356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:50.809400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:50.942688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01707a3-1c14-4911-817f-b6a8eadd328f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:50.947853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:50.948467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:50.948741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:50.949006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:50.970632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:50.970632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:50.976302 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1334/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:54:51.424045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:51.426189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:51.501299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a80a05-1d17-4211-82bc-534de2b5687f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:51.506385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:51.507033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:51.507315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:51.507588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:51.509523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:51.636223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:51.638050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:51.653266 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1388/2722] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:51.845688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:51.847431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:51.861479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51cdd3e-b80d-4c09-8c20-7be4b7d635d1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:51.880632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:51.881294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:51.881576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:51.882163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] PATH_INFO: `/v3/projects/6d21d03d92e1426f8043940883de088c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:51.883249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d21d03d92e1426f8043940883de088c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:51.885172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:51.935703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d21d03d92e1426f8043940883de088c', 'request_id': 'req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:51.943364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d21d03d92e1426f8043940883de088c', 'request_id': 'req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:51.951379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d21d03d92e1426f8043940883de088c', 'request_id': 'req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:51.963619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] The token cache is being invalidate because project 6d21d03d92e1426f8043940883de088c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:51.964091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:51.964883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:51.967109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1335/2723] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:51 2022] DELETE /identity/v3/projects/6d21d03d92e1426f8043940883de088c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:51.971205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:51.983318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e6d91-ee2d-4846-8423-8e2738cdc5c1 service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:52.028394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:52.063845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:52.064475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:52.064755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:52.065026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] PATH_INFO: `/v3/users/4937765d76e44f1ca2f3390f4f2e5966` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:52.066415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4937765d76e44f1ca2f3390f4f2e5966)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:52.069451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:52.132914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4937765d76e44f1ca2f3390f4f2e5966'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.133612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4937765d76e44f1ca2f3390f4f2e5966'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.143280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4937765d76e44f1ca2f3390f4f2e5966'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.161110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4937765d76e44f1ca2f3390f4f2e5966'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.161928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54e76657-1429-4d5f-8b98-94cd90b6cea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4937765d76e44f1ca2f3390f4f2e5966'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.186086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1389/2724] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:51 2022] DELETE /identity/v3/users/4937765d76e44f1ca2f3390f4f2e5966 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:52.621135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:52.625278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:52.650175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:52.662975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ec6bb-1bd6-42da-ada2-c1668db6b3fc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:52.727895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:52.728615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:52.728957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:52.729310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] PATH_INFO: `/v3/projects/d802560be740486e946ded17fbf68906` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:52.730530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d802560be740486e946ded17fbf68906)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:52.732525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:52.804062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd802560be740486e946ded17fbf68906', 'request_id': 'req-26539d7e-63b5-48ef-8cd6-8688866b7ec6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.810602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd802560be740486e946ded17fbf68906', 'request_id': 'req-26539d7e-63b5-48ef-8cd6-8688866b7ec6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.832618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd802560be740486e946ded17fbf68906', 'request_id': 'req-26539d7e-63b5-48ef-8cd6-8688866b7ec6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.849460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] The token cache is being invalidate because project d802560be740486e946ded17fbf68906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:52.850012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.850898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26539d7e-63b5-48ef-8cd6-8688866b7ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:52.852881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1336/2725] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:52 2022] DELETE /identity/v3/projects/d802560be740486e946ded17fbf68906 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:52.858690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:52.859633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:52.861468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:52.889915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:52.891260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:52.968200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1390/2726] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:52.978393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:52.978928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:52.979131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.000634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:53.002078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:53.010774 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1337/2727] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:53.014982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.026748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3706ec-1d80-404c-ac31-01a627a31f72 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.031482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.032090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.032367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.032638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.033283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:53.033586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.035951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.048101 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1391/2728] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:53.053006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.062242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61085d0-f666-45a6-964c-5381010eeff1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.067409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.068024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.068302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.068577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.069217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.071182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.100580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1338/2729] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:53.104366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.114541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcae49-a7b5-43a9-936f-ab4edf267a99 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.166874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:53.184614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.185220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.185501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.185776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.186406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.188712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.214487 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1392/2730] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:53.219174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.235862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5abb11-a7b5-4a12-b388-0c2f31c2d28d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.241338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.242310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.242310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.242622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.243261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.244851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.257414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1339/2731] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:53.261989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.274912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b39b3-8612-484d-9277-7225252c7808 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.279894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.281340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.281696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.282042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.282796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.285930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88111f-59ca-42f1-845c-beca9303dc14 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.297889 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1393/2732] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:53.302946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.313063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb137f-ea0e-4280-b86c-1afd4942ae15 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.320237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.320978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.321330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.321792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] PATH_INFO: `/v3/projects/c981726b84d042d9a9e3984610185bbe/users/f86860117c4c4a8881ab0deefa34e1bf/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.337250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c981726b84d042d9a9e3984610185bbe, user_id=f86860117c4c4a8881ab0deefa34e1bf, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.339995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.356571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1340/2733] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:53 2022] PUT /identity/v3/projects/c981726b84d042d9a9e3984610185bbe/users/f86860117c4c4a8881ab0deefa34e1bf/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:53.371537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.371827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.372155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.401316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] MFA Rules not processed for user `f86860117c4c4a8881ab0deefa34e1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:53.402596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:53.468273 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1394/2734] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:53.473337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.506435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:53.537664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7f31f-a041-4057-a996-61a2aee057c7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.542487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.543107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.543385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.543655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.544267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.546176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.573415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1341/2735] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:53.577817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.588651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2618c7-37b6-49b3-b362-a70ef8f1cc30 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.645231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.645861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.646119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.646386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.647039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.649197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.672510 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1395/2736] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:53.677073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.687573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a10b-6468-4418-b257-68a77894b114 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.693003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.693617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.693895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.694168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.694917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.696545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.708753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1342/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:53.713185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.726079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b0ba9-8ead-4b9f-933d-59a090855c60 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.730727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.731378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.731655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.731994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] PATH_INFO: `/v3/projects/15a18f2f3eb940f7a04196480721613d/users/ab0642d5979b418384f449e58c2c69ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.747991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a18f2f3eb940f7a04196480721613d, user_id=ab0642d5979b418384f449e58c2c69ba, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.750704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.764028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1396/2738] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:53 2022] PUT /identity/v3/projects/15a18f2f3eb940f7a04196480721613d/users/ab0642d5979b418384f449e58c2c69ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:53.767939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.779171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827fb2-2adb-4db3-9359-12bf25214ef4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.844468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.851080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.851529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.852025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.853903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.855938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e91a5d-08e4-4ff4-9ed1-f4e6165ebfda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.869701 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1343/2739] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:53.870689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:53.881306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde312e-b161-4f38-b804-1a8e21d89dd2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:53.885936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:53.886619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.886897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.887161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] PATH_INFO: `/v3/projects/15a18f2f3eb940f7a04196480721613d/users/ab0642d5979b418384f449e58c2c69ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.889080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a18f2f3eb940f7a04196480721613d, user_id=ab0642d5979b418384f449e58c2c69ba, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:53.891742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:53.903180 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1397/2740] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:53 2022] PUT /identity/v3/projects/15a18f2f3eb940f7a04196480721613d/users/ab0642d5979b418384f449e58c2c69ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:53.914344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:53.914680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:53.914964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:53.945927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] MFA Rules not processed for user `ab0642d5979b418384f449e58c2c69ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:53.947168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:54.030093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1344/2741] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:54.037742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.044367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:54.049029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bb77e-8095-48a2-a014-2fae998bbbea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.055317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.055921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.056194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.056462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.057122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:54.057421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.059047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.073590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1398/2742] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.077427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.087969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443100d3-d826-450a-95e2-66fefbaabc8a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.093345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.093946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.094228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.094519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.095154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.097746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.127756 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1345/2743] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:54 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:54.131755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.142264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d807a88-82e2-4628-9b9e-826ff5617bbd tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.207154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.207764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.208045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.208321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.208960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:54.209282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.210993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4183ef-fbf6-437c-aadb-915dbd79a052 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.226440 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1399/2744] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.234146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.235961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:54.243562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:54.313833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70190dad-d708-4d7c-bee0-36256d214373 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.320342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.320951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.321229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.321500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.330104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.332693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.343735 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1346/2745] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:54.372118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44161f7-29f0-4155-95e0-7a14cb7e5dc9 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.372414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44161f7-29f0-4155-95e0-7a14cb7e5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.372685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44161f7-29f0-4155-95e0-7a14cb7e5dc9 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.374321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1400/2746] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:54:54.381993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:54:54.394205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.403756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e75595-cfc1-49fd-a176-7da18fab18e2 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.408545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.409991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.410305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.410614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.411899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:get_project(project_id=c2cc93bc366345b2853c71bcbd47dcf6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.414683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.416367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1347/2747] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6 => generated 369 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.580604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d85a8-abff-4e53-a568-3016f5e3cfe5 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.581536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d85a8-abff-4e53-a568-3016f5e3cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.582271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d85a8-abff-4e53-a568-3016f5e3cfe5 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.583799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1401/2748] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:54:54.589273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:54:54.591104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:54.607973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.617761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bffc7-bfdf-4ac7-8d27-ccce004877ee tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.622575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.623291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.623647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.623994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.625130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:get_project(project_id=c2cc93bc366345b2853c71bcbd47dcf6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.627262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.635177 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1348/2749] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6 => generated 369 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.685476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d85a8-abff-4e53-a568-3016f5e3cfe5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.712585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d85a8-abff-4e53-a568-3016f5e3cfe5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.727635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.728252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.728533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.728806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.729454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:54.729762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.731790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.746540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1402/2750] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.749533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.760632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a2487-e629-48f8-be2e-363a23c0d054 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.765314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.765899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.766176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.766448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.767120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.770373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999d187-2a72-4816-8bf1-5e23f9c29bf2 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.793219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1349/2751] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:54 2022] POST /identity/v3/users => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:54.797380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.807550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19294f11-a7df-4e17-aded-bc8583798467 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.812447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.813049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.813327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.813599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.814235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:54.814572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.816386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.830531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1403/2752] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:54.848649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a07a9-0968-4406-8c05-18adc4773cd4 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.848933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a07a9-0968-4406-8c05-18adc4773cd4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.849207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a07a9-0968-4406-8c05-18adc4773cd4 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.850555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1350/2753] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:54:54.858585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:54:54.875000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:54.885000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fafbf-96e0-44d1-ac61-4e807c5a3f7b tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:54.889877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:54.890540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:54.890819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:54.891091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:54.892234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:get_project(project_id=c2cc93bc366345b2853c71bcbd47dcf6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:54.895178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:54.897429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1404/2754] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:54:54 2022] GET /identity/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6 => generated 369 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:55.125683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017ff8-2460-4fbb-be76-d8a570758c46 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:55.126219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017ff8-2460-4fbb-be76-d8a570758c46 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:55.126613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017ff8-2460-4fbb-be76-d8a570758c46 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:55.128760 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1351/2755] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 20:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 20:54:55.136745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 20:54:55.163638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:55.180029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c174cad-d54a-4af3-a86c-21abf0c8858d tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:55.207274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:55.208049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:55.208333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:55.208609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:55.212322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:get_project(project_id=c2cc93bc366345b2853c71bcbd47dcf6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:55.219738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:55.249497 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1405/2756] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 20:54:55 2022] GET /identity/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6 => generated 369 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:55.293818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017ff8-2460-4fbb-be76-d8a570758c46 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:55.312271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017ff8-2460-4fbb-be76-d8a570758c46 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:55.324917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:55.325861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:55.326208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:55.326576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/users/3d83fecb4f8b467b8fe86558fc6ec2ba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:55.390296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:delete_user(user_id=3d83fecb4f8b467b8fe86558fc6ec2ba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:55.391246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:55.460849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d83fecb4f8b467b8fe86558fc6ec2ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.461530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d83fecb4f8b467b8fe86558fc6ec2ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.473512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d83fecb4f8b467b8fe86558fc6ec2ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.482908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d83fecb4f8b467b8fe86558fc6ec2ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.483640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d83fecb4f8b467b8fe86558fc6ec2ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.507389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1352/2757] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:55 2022] DELETE /identity/v3/users/3d83fecb4f8b467b8fe86558fc6ec2ba => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:55.511473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:55.513352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:55.527699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c78ff5-68fc-4a7a-981e-a4f3b4041b40 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:55.606236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab0642d5979b418384f449e58c2c69ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '15a18f2f3eb940f7a04196480721613d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:55.606927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:55.607501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:55.607780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] PATH_INFO: `/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:55.609843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c2cc93bc366345b2853c71bcbd47dcf6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:55.611872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:55.671759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:55.677190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cc93bc366345b2853c71bcbd47dcf6', 'request_id': 'req-45420901-58fb-4879-9dd4-2d57feac9670'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.686004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cc93bc366345b2853c71bcbd47dcf6', 'request_id': 'req-45420901-58fb-4879-9dd4-2d57feac9670'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.694511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cc93bc366345b2853c71bcbd47dcf6', 'request_id': 'req-45420901-58fb-4879-9dd4-2d57feac9670'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.710951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] The token cache is being invalidate because project c2cc93bc366345b2853c71bcbd47dcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:55.711463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.712523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-45420901-58fb-4879-9dd4-2d57feac9670 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:55.714097 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1406/2758] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:55 2022] DELETE /identity/v3/projects/c2cc93bc366345b2853c71bcbd47dcf6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:55.719486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:55.722490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:55.743880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:55.855876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b94d52-0a8f-42b6-a030-5e7c850cb3c0 tempest-QuotasAdminTestV236-774709866 tempest-QuotasAdminTestV236-774709866-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:55.948449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:55.949095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:55.949373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:55.950021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] PATH_INFO: `/v3/users/f86860117c4c4a8881ab0deefa34e1bf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:55.950920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86860117c4c4a8881ab0deefa34e1bf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:55.953219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:56.017285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86860117c4c4a8881ab0deefa34e1bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.017985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86860117c4c4a8881ab0deefa34e1bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.028059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86860117c4c4a8881ab0deefa34e1bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.038079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86860117c4c4a8881ab0deefa34e1bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.038951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86860117c4c4a8881ab0deefa34e1bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.057954 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1353/2759] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:55 2022] DELETE /identity/v3/users/f86860117c4c4a8881ab0deefa34e1bf => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:56.062407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:56.062729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:56.063006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:56.084142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:56.085371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:56.202391 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1407/2760] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:56.221035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:56.223361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:56.290773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:56.308600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044b64c-96ed-48e3-ae05-429af1740715 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:56.313934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:56.314720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:56.315189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:56.316490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:56.328329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:56.329994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:56.352125 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1354/2761] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:54:56.800789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:56.802398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:56.814668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a39683-6dbe-4a26-9ce4-bb70989d68d6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:56.830231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:56.834568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:56.834568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:56.834777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] PATH_INFO: `/v3/projects/c981726b84d042d9a9e3984610185bbe` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:56.836141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c981726b84d042d9a9e3984610185bbe)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:56.838435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:56.891761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c981726b84d042d9a9e3984610185bbe', 'request_id': 'req-74c9296a-f290-40c5-888a-71e19dce7393'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.900885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c981726b84d042d9a9e3984610185bbe', 'request_id': 'req-74c9296a-f290-40c5-888a-71e19dce7393'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.914081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c981726b84d042d9a9e3984610185bbe', 'request_id': 'req-74c9296a-f290-40c5-888a-71e19dce7393'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.930053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] The token cache is being invalidate because project c981726b84d042d9a9e3984610185bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:56.930363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.931045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:56.932862 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1408/2762] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:56 2022] DELETE /identity/v3/projects/c981726b84d042d9a9e3984610185bbe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:56.936891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:56.949428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d1bed-5cd8-4beb-a8b0-ead6e4fc8ae9 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:57.023186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:57.024160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.024516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.024856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] PATH_INFO: `/v3/users/ab0642d5979b418384f449e58c2c69ba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.026047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0642d5979b418384f449e58c2c69ba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:57.028225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:57.076646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0642d5979b418384f449e58c2c69ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.077214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0642d5979b418384f449e58c2c69ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.085094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0642d5979b418384f449e58c2c69ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.092110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0642d5979b418384f449e58c2c69ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.092737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-477ae5b7-c14e-4b59-a5c0-36bc6dd7b83a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0642d5979b418384f449e58c2c69ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.110257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1355/2763] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:54:56 2022] DELETE /identity/v3/users/ab0642d5979b418384f449e58c2c69ba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:57.395838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:57.397301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:57.413063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9296a-f290-40c5-888a-71e19dce7393 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:57.467808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:57.477067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:57.477739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.478180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.478457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] PATH_INFO: `/v3/projects/15a18f2f3eb940f7a04196480721613d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.479661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a18f2f3eb940f7a04196480721613d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:57.481590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:57.527733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a18f2f3eb940f7a04196480721613d', 'request_id': 'req-a71606c3-5b48-4e2c-82ea-ef538b34ab54'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.534447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a18f2f3eb940f7a04196480721613d', 'request_id': 'req-a71606c3-5b48-4e2c-82ea-ef538b34ab54'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.541237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a18f2f3eb940f7a04196480721613d', 'request_id': 'req-a71606c3-5b48-4e2c-82ea-ef538b34ab54'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.555052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] The token cache is being invalidate because project 15a18f2f3eb940f7a04196480721613d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:54:57.555342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.555979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a71606c3-5b48-4e2c-82ea-ef538b34ab54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:54:57.557715 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1409/2764] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:54:57 2022] DELETE /identity/v3/projects/15a18f2f3eb940f7a04196480721613d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:57.564174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdd765-7f63-4146-b6e5-bf501d803a3c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.564621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdd765-7f63-4146-b6e5-bf501d803a3c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.564811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdd765-7f63-4146-b6e5-bf501d803a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.585292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e7bdd765-7f63-4146-b6e5-bf501d803a3c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:57.586522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e7bdd765-7f63-4146-b6e5-bf501d803a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:57.638450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.638776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.639059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.670361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] MFA Rules not processed for user `9c03c67e0c4c4de7a186f28ac10d0b92`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:57.671664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:57.679252 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1356/2765] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:57.687358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.687641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.687915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.699139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:57.712043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:57.714522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:57.728437 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1357/2766] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:57.732830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:57.736177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1410/2767] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:54:57.744825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597d3ee-1320-4b0f-b48b-c08f932103ae None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:57.751057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:57.751834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.752426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.754706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.755419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:54:57.755730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:57.757344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:57.777808 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1358/2768] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:54:57.782141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:57.798348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc471fc4-6564-4fd8-b4f6-72ae97757b86 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:57.804554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:57.805317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.805675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.806163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.806930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:57.808840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:57.853174 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1411/2769] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:57 2022] POST /identity/v3/projects => generated 395 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:57.857323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:57.882296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf7d1d-68c7-479c-b22e-359f98fd7437 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:57.969298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:57.969921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:57.970174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:57.970447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:57.971949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:57.973758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:57.996733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1359/2770] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:57 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:58.001276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.013001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8c4e9-e3e5-4d58-9e0b-d48ffcf753fb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.020433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.020433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.020433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.020433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.020834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.022423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.034860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1412/2771] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:58.039102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.053200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa175154-bcbe-4ba3-b0d7-fd88c16bf574 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.060877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.062130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.062648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.063106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.063908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.066952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfde49-5db6-408c-aff2-fd95bb93996f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.086996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1360/2772] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:58.088009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.098934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d26a61-22c3-4940-89dd-354c95e4d8fa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.106265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.107044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.107325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.107595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] PATH_INFO: `/v3/projects/dccf2c674b1748c6b6e0b33e05e02940/users/8a0244a0ef19409a8a6e7c6526685919/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.123665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccf2c674b1748c6b6e0b33e05e02940, user_id=8a0244a0ef19409a8a6e7c6526685919, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.126324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.146990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1413/2773] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:58 2022] PUT /identity/v3/projects/dccf2c674b1748c6b6e0b33e05e02940/users/8a0244a0ef19409a8a6e7c6526685919/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:54:58.159561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.160053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.160405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.196305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] MFA Rules not processed for user `8a0244a0ef19409a8a6e7c6526685919`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:58.197745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:58.276206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1361/2774] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:58.281780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.323024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:58.361121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076d045-2fbc-4345-9d03-5f27e29ab2c8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.367154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.367750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.368241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.368467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.368975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.371061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.406167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1414/2775] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:54:58 2022] POST /identity/v3/projects => generated 395 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:54:58.411033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.422906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03e31b-807a-4f27-b5dc-0f78a6d30ba4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.479897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:58.492325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.493097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.493481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.493818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.494550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.496677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.522873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1362/2776] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:54:58 2022] POST /identity/v3/users => generated 304 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:54:58.526821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.541013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8096a8d-66c7-4b82-b041-9320337ff48d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.546285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.546910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.547187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.547452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.548140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.551002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.564667 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1415/2777] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:54:58.569381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.580238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec49108-534c-4fb3-aa48-c97e6e119638 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.585702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.586565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.586956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.587379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] PATH_INFO: `/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd/users/dea41ec854054a768ad25e9ac6a28f71/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.603314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc32e5dfc1f446f6a095c10e5363bcbd, user_id=dea41ec854054a768ad25e9ac6a28f71, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.606297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.620361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1363/2778] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:58 2022] PUT /identity/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd/users/dea41ec854054a768ad25e9ac6a28f71/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:58.624731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.636287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db826-5b43-42a4-80d1-7321f3395e90 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.697769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.698448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.698764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.699033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.699670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.701266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d6cbb-bebd-4acd-a2b8-eba5f1f6dfb6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.711621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1416/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:54:58.715841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:58.729641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63bc9f-31dc-4c24-9ae0-25b9e7a51e5c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:58.734517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:58.735192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.735461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.735723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] PATH_INFO: `/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd/users/dea41ec854054a768ad25e9ac6a28f71/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.738696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc32e5dfc1f446f6a095c10e5363bcbd, user_id=dea41ec854054a768ad25e9ac6a28f71, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:58.741381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4abe4c-297e-48a1-8820-8df2cd75ae72 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:58.752210 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1364/2780] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:54:58 2022] PUT /identity/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd/users/dea41ec854054a768ad25e9ac6a28f71/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:54:58.762707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.762986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.763262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.795434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] MFA Rules not processed for user `dea41ec854054a768ad25e9ac6a28f71`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:58.796694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:58.881054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1417/2781] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:58.958583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:58.959244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:58.959527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:58.981960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] MFA Rules not processed for user `8a0244a0ef19409a8a6e7c6526685919`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:54:58.983230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:59.072392 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1365/2782] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:54:59.082991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:54:59.085786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:54:59.124411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:54:59.176001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2a6e-f8b2-4418-8728-b12dd33bf331 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:54:59.183657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:54:59.184278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:54:59.184564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:54:59.184839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:54:59.198463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:54:59.200261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244c18f-460e-4e46-970b-1677838f55b8 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:54:59.213035 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1418/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:00.104721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:00.106168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:00.233747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c613e-b1c0-45ad-9bdc-0e9b7edef8e8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:00.256258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:00.256926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:00.257210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:00.257710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:00.259895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:00.279182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:00.282589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:00.302736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1366/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:01.411352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:01.411951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:01.412058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:01.443275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:01.445304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:01.565920 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1419/2785] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:55:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:01.594922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:01.597128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:01.682384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:01.732545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6e5f-1a39-4bb2-a5b3-2a7f2418f12a service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:01.739914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:01.741248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:01.741793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:01.742279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:01.754595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:01.756255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:01.778910 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1367/2786] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:55:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:55:07.884199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:07.885818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:07.974730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:07.976517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:08.004105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f3f68-9eec-4323-9d74-e6d275e74e03 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:08.016905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:08.017657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:08.018033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:08.018990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:08.029696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc801c-f76b-448f-8c7c-56a8f8f174f9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:08.035498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:08.036662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:08.037612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:08.037982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:08.038430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:08.040953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:08.055206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:08.057676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:08.064237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:08.067089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:08.072263 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1420/2787] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:08.078675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1368/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:08.567642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:08.593128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664161-1363-46e7-bc12-d1252d9590d7 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:08.605760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:08.606440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:08.606786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:08.607059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:08.611278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:08.640877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:08.646098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:08.677914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1421/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:14.896936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:14.912989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce801125-142e-4f7b-9397-b41b8089e227 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:14.924768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:14.925817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:14.925916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:14.926682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:14.929390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:15.023108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:15.025166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e3125-af7f-4a5b-97cf-7bae6f333848 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:15.066250 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1369/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:18.654268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=78 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:19.592267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:19.593776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:19.668805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2f0b-b3b1-435a-b439-c13f5ded23eb service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:19.673395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:19.674022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:19.674301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:19.675184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] PATH_INFO: `/v3/users/2844f27200534323910d80cd869a6601` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:19.676506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2844f27200534323910d80cd869a6601)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:19.678417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:19.726166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2844f27200534323910d80cd869a6601'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:19.726775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2844f27200534323910d80cd869a6601'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:19.734364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2844f27200534323910d80cd869a6601'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:19.747334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2844f27200534323910d80cd869a6601'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:19.747981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2844f27200534323910d80cd869a6601'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:19.760441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1422/2791] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:19 2022] DELETE /identity/v3/users/2844f27200534323910d80cd869a6601 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:19.765082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:19.765604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:19.765742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:19.785905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:19.787125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:19.847450 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1370/2792] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:19.856667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:19.858085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:19.948815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe4c09-545f-4bb5-a034-d1cbd04551ad admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:19.954186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:19.955167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:19.955639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:19.956341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:19.968327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:19.970294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:19.983324 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1423/2793] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:20.171068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:20.173214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:20.208130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b0ec7-b4ca-4d31-88ad-e22d534669c8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:20.221118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:20.224433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:20.225132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:20.225746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] PATH_INFO: `/v3/projects/4b55ef17d925408093f5407fa8015787` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:20.229109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b55ef17d925408093f5407fa8015787)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:20.233590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:20.316227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b55ef17d925408093f5407fa8015787', 'request_id': 'req-58a92a68-7f97-4163-8b10-c0e88ef0c265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.321349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:20.335833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b55ef17d925408093f5407fa8015787', 'request_id': 'req-58a92a68-7f97-4163-8b10-c0e88ef0c265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.352896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b55ef17d925408093f5407fa8015787', 'request_id': 'req-58a92a68-7f97-4163-8b10-c0e88ef0c265'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.367941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] The token cache is being invalidate because project 4b55ef17d925408093f5407fa8015787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:20.368232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.368906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-58a92a68-7f97-4163-8b10-c0e88ef0c265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.373875 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1371/2794] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:20 2022] DELETE /identity/v3/projects/4b55ef17d925408093f5407fa8015787 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:20.375208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:20.391687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a4bdb-8bd2-4a60-8cbf-ab1e9a608869 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:20.461371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:20.699575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:20.700718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:20.701161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:20.701570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] PATH_INFO: `/v3/users/0f596f58695644da8f20168715392c49` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:20.710960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f596f58695644da8f20168715392c49)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:20.714719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:20.818834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f596f58695644da8f20168715392c49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.819563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f596f58695644da8f20168715392c49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.835753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f596f58695644da8f20168715392c49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.850973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f596f58695644da8f20168715392c49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.851769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f596f58695644da8f20168715392c49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:20.866590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1424/2795] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:20 2022] DELETE /identity/v3/users/0f596f58695644da8f20168715392c49 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:20.984785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:20.986056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:20.986896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:21.028702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:21.029984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:21.172181 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1372/2796] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:55:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:21.189102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:21.189996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:21.386769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:21.388526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b10a0-78ce-4557-aab0-21b68ed6b72c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:21.388886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:21.403964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:21.405020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:21.405414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:21.405691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:21.455829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:21.478292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8749cbb-81d9-4e1a-9575-3b36193d67f0 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:21.520965 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1425/2797] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:55:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:55:21.556245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18983af-c88b-47f0-bfe9-216c4cb62b83 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:21.589875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:21.599308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:21.599879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:21.600339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] PATH_INFO: `/v3/projects/2464ee933b394763a0135453d1072a2d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:21.602016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2464ee933b394763a0135453d1072a2d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:21.604243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:21.781550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2464ee933b394763a0135453d1072a2d', 'request_id': 'req-1578af41-cb7b-495a-87ee-92c9b4572df8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:21.791098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2464ee933b394763a0135453d1072a2d', 'request_id': 'req-1578af41-cb7b-495a-87ee-92c9b4572df8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:21.827296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2464ee933b394763a0135453d1072a2d', 'request_id': 'req-1578af41-cb7b-495a-87ee-92c9b4572df8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:21.856394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] The token cache is being invalidate because project 2464ee933b394763a0135453d1072a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:21.856947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:21.858441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1578af41-cb7b-495a-87ee-92c9b4572df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:21.869527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1373/2798] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:21 2022] DELETE /identity/v3/projects/2464ee933b394763a0135453d1072a2d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:21.871393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:21.871899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:21.871984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:21.923796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:21.925252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:22.025647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:22.114577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1426/2799] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:22.195410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.196274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.196711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.278719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:22.280251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:22.290696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.292971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:22.300677 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1374/2800] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:22.304864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.321854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef160f-4d3b-447d-a27f-83d4d162ae56 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.328208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.328852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.329133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.329465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.330082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:22.330645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.332956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.334953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:22.347956 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1375/2801] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:55:22.352200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.388805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590097f-2514-4b3e-bba9-26f6497b0733 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.400187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.402305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.402773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.403647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.405565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.414468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.481183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1376/2802] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:22 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:22.489955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.500915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c38510-e768-4781-86c6-609ce3b1f3c9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.671158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.671348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed43a9-1185-4f33-8e23-46365102daa4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.672133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.672645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.673021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.676023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:22.684448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.685070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.685350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.685619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.686381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.691452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.691589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.694105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.709747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1427/2803] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:55:22.722219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1377/2804] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:22 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:22.724073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.740382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195b2b-1491-43d8-85c9-db3ebe306c57 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.746665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.747335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.747619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.747890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.748541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.750654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.769478 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1428/2805] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:22.773359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.787747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228aaf-5eec-48fc-a38b-dc566037fa54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.793181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.799464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.801851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.802295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.803347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.805986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02729c-1c95-4483-9ce5-daf2be289a0f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.824209 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1378/2806] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:22.828966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:22.844385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e251c00-8ca8-4eef-b1b3-1ca89bb27205 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:22.850957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:22.851913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.852254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.852655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] PATH_INFO: `/v3/projects/3a215e5d0f694eb28b015a77a65fed65/users/9f84e590b82949999943ec418f5ce4f5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.875081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a215e5d0f694eb28b015a77a65fed65, user_id=9f84e590b82949999943ec418f5ce4f5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:22.877840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:22.896254 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1429/2807] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:22 2022] PUT /identity/v3/projects/3a215e5d0f694eb28b015a77a65fed65/users/9f84e590b82949999943ec418f5ce4f5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:22.907073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d35b-da67-4407-9573-765c06345182 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:22.907411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d35b-da67-4407-9573-765c06345182 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:22.907629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d35b-da67-4407-9573-765c06345182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:22.945401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0004d35b-da67-4407-9573-765c06345182 None None] MFA Rules not processed for user `9f84e590b82949999943ec418f5ce4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:22.947184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0004d35b-da67-4407-9573-765c06345182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.036678 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1379/2808] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:23.041486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.080062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:23.157710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34c73-875c-4f1a-b0d6-3233de7449e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.164140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.164759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.165039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.165311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.166004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.168272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.201868 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1430/2809] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:23 2022] POST /identity/v3/projects => generated 395 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:23.207951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004d35b-da67-4407-9573-765c06345182 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.239096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004d35b-da67-4407-9573-765c06345182 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.316431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:23.343827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.344549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.344899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.345319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.345955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.349057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.373497 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1380/2810] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:23 2022] POST /identity/v3/users => generated 304 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:23.378864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.390365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980c0bb-7f0d-4afe-9397-3d8c56991541 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.396565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.397364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.397745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.398133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.398917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.400910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.414191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1431/2811] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:23.418277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.430613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14596485-7617-4099-abbe-e183240c3ec3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.435949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.436628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.436975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.437227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] PATH_INFO: `/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c/users/2a7fd61a8e8b441396226391a9e6c79d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.455525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e829a2e0f6247a5b1b5faac26d4028c, user_id=2a7fd61a8e8b441396226391a9e6c79d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.459158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.475213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1381/2812] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:23 2022] PUT /identity/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c/users/2a7fd61a8e8b441396226391a9e6c79d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:23.479353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.491481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d2047-ef04-4a2c-895b-eb653334c0dc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.527247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.529634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.565475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.566298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.566661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.566877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.567697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.569870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.582780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1432/2813] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:23.585740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.596165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988f213-3fec-447c-bc57-6657d5bbb35a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.602065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.603039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.603256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.603594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] PATH_INFO: `/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c/users/2a7fd61a8e8b441396226391a9e6c79d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.606164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e829a2e0f6247a5b1b5faac26d4028c, user_id=2a7fd61a8e8b441396226391a9e6c79d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.608643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b463e-60aa-4511-a20a-aaec466ea6d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.609012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434d06-d311-412f-9b7a-db73509b7738 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.614449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.615189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.615539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.615880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] PATH_INFO: `/v3/users/8a0244a0ef19409a8a6e7c6526685919` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.617088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0244a0ef19409a8a6e7c6526685919)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:23.619135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:23.621764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1433/2814] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:23 2022] PUT /identity/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c/users/2a7fd61a8e8b441396226391a9e6c79d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:23.635844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.636429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.636969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.671303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] MFA Rules not processed for user `2a7fd61a8e8b441396226391a9e6c79d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:23.672689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.685040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0244a0ef19409a8a6e7c6526685919'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:23.685716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0244a0ef19409a8a6e7c6526685919'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:23.704287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0244a0ef19409a8a6e7c6526685919'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:23.717339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0244a0ef19409a8a6e7c6526685919'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:23.718177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-80e9af7c-8645-47b8-ade8-4963ae2a75e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0244a0ef19409a8a6e7c6526685919'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:23.753460 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1382/2815] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:23 2022] DELETE /identity/v3/users/8a0244a0ef19409a8a6e7c6526685919 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:23.754967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169792d2-abef-4602-ae23-866689be689c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.755253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169792d2-abef-4602-ae23-866689be689c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.755525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169792d2-abef-4602-ae23-866689be689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:23.769385 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1434/2816] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:23.777713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.779113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.781464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-169792d2-abef-4602-ae23-866689be689c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:23.782837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-169792d2-abef-4602-ae23-866689be689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.784848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:23.808696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:23.888216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1383/2817] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:23.894949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169792d2-abef-4602-ae23-866689be689c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:23.895670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78b43-7785-401f-8d46-aa26ec633764 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:23.896447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-169792d2-abef-4602-ae23-866689be689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:23.995079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:23.995691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:23.995966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:23.996234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.000125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169792d2-abef-4602-ae23-866689be689c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.005637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.006808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.007383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.007877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.025283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.028283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.059302 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1384/2818] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:55:24.096667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.098803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.126353 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1435/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:55:24.199908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:24.222313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbee3-1f73-4165-9a5b-f75fd2ec509d service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.240969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.241738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.242094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.242437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.248570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:24.266073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.270270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.283733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1385/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:24.507056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:24.508836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:24.536436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22c84b-ae9e-4f3c-9886-5334180b2f4f service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.548798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.549443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.550576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.550866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] PATH_INFO: `/v3/projects/dccf2c674b1748c6b6e0b33e05e02940` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.555346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccf2c674b1748c6b6e0b33e05e02940)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.557281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.624038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:24.654467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccf2c674b1748c6b6e0b33e05e02940', 'request_id': 'req-e31cda79-d766-45c8-b547-e5c20cd3dc11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.663612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccf2c674b1748c6b6e0b33e05e02940', 'request_id': 'req-e31cda79-d766-45c8-b547-e5c20cd3dc11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.677521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:24.683856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccf2c674b1748c6b6e0b33e05e02940', 'request_id': 'req-e31cda79-d766-45c8-b547-e5c20cd3dc11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.689180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574062bb-4850-455c-ad6f-58cde861be62 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.699835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] The token cache is being invalidate because project dccf2c674b1748c6b6e0b33e05e02940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:24.700163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.700814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.702367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1436/2821] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:24 2022] DELETE /identity/v3/projects/dccf2c674b1748c6b6e0b33e05e02940 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:24.706857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:24.708232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:24.773144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.773897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.774221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.774505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.776574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:24.790080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cda79-d766-45c8-b547-e5c20cd3dc11 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.795928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.796568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.796849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.797118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] PATH_INFO: `/v3/users/dea41ec854054a768ad25e9ac6a28f71` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.798298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea41ec854054a768ad25e9ac6a28f71)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.801827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.836615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:24.849345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.851401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:24.856247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea41ec854054a768ad25e9ac6a28f71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.857313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea41ec854054a768ad25e9ac6a28f71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.864693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1386/2822] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:24.867557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea41ec854054a768ad25e9ac6a28f71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.884499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea41ec854054a768ad25e9ac6a28f71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.884499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-62991296-54af-4137-a8da-b2fad63f8c36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea41ec854054a768ad25e9ac6a28f71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:24.892955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:24.895210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:24.902087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1437/2823] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:24 2022] DELETE /identity/v3/users/dea41ec854054a768ad25e9ac6a28f71 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:24.910291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c62c38-56a1-4941-8068-be79bb32b020 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:24.981053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:24.981682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:24.981959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:24.982227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] PATH_INFO: `/v3/users/9f84e590b82949999943ec418f5ce4f5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:24.983454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f84e590b82949999943ec418f5ce4f5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:24.985673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.056721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f84e590b82949999943ec418f5ce4f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.058328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f84e590b82949999943ec418f5ce4f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.067451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f84e590b82949999943ec418f5ce4f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.086109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f84e590b82949999943ec418f5ce4f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.087126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f84e590b82949999943ec418f5ce4f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.110893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1387/2824] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:24 2022] DELETE /identity/v3/users/9f84e590b82949999943ec418f5ce4f5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:25.113703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.114625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.114935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.137385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:25.138641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.174684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:25.225433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.228308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.245376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1438/2825] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:25.246394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.247872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.264253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9cc00-67de-4ffc-8703-09b542265d64 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.271661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.273270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.273780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.274151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] PATH_INFO: `/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.276434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc32e5dfc1f446f6a095c10e5363bcbd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.278476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.327660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787f1a2-548e-4a5f-9c7b-1152156a19dd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.351088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc32e5dfc1f446f6a095c10e5363bcbd', 'request_id': 'req-b7e3f828-de4b-493f-ad4b-889efc0b74dd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.366680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc32e5dfc1f446f6a095c10e5363bcbd', 'request_id': 'req-b7e3f828-de4b-493f-ad4b-889efc0b74dd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.381807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc32e5dfc1f446f6a095c10e5363bcbd', 'request_id': 'req-b7e3f828-de4b-493f-ad4b-889efc0b74dd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.385437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:25.431806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] The token cache is being invalidate because project dc32e5dfc1f446f6a095c10e5363bcbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:25.433413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.434522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b7e3f828-de4b-493f-ad4b-889efc0b74dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:25.446823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1388/2826] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:25 2022] DELETE /identity/v3/projects/dc32e5dfc1f446f6a095c10e5363bcbd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:25.453663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63296c0-8edc-449b-81d5-db37de15866d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.453947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63296c0-8edc-449b-81d5-db37de15866d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.454218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63296c0-8edc-449b-81d5-db37de15866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.474934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.476094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.476476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.476901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.479554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.503615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e63296c0-8edc-449b-81d5-db37de15866d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:25.504948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e63296c0-8edc-449b-81d5-db37de15866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.576390 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1389/2827] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:25.578020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.579712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.584674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.585240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.585930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.591045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1439/2828] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:25.611659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:25.614599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:25.626800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1390/2829] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:25.630545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.645802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f08c-52b4-4933-b22c-afc145608364 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.664701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.665972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.666252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.666555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.667219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:25.667525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.669330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.686728 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1440/2830] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:25.689463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.700533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9645e1-747e-442c-8ddb-019937155546 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.706069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.706715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.707025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.707252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.708017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.709793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.752430 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1391/2831] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:25 2022] POST /identity/v3/projects => generated 397 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:25.757525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.775436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b7f1f-2da5-44be-9f63-3a45329f618b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.809309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:25.856966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.857656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.857938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.858212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.858977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.861419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.890777 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1441/2832] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:25 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:25.896516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.909480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326921d-6a93-4eb0-8833-c62c88d30d6f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.915010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.915777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.916122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.916471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.917277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.919085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.945365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:25.946665 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1392/2833] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:25.959602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed41b5-3c25-4650-88fa-6a16f2b52b15 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:25.965238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:25.965966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:25.966316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:25.966693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:25.967437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:25.969209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32d0b3-4b94-4278-918b-4917a79937ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:25.983668 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1442/2834] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:25.996790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:26.022273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84286b-980d-4a72-8725-f03f911e1f3a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:26.028787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:26.028787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:26.029132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:26.029227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] PATH_INFO: `/v3/projects/2b59c07031e64582ac08d14f9aa85a68/users/36cada5c067342218850b5d13a17627c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:26.077890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b59c07031e64582ac08d14f9aa85a68, user_id=36cada5c067342218850b5d13a17627c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:26.081535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:26.115635 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1393/2835] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:25 2022] PUT /identity/v3/projects/2b59c07031e64582ac08d14f9aa85a68/users/36cada5c067342218850b5d13a17627c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:26.126462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:26.127521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:26.128366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:26.196280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] MFA Rules not processed for user `36cada5c067342218850b5d13a17627c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:26.197830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:26.295496 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1443/2836] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:26 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:26.308159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:26.309983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:26.450889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8eb-58d8-4977-bfd9-2eb26eb61cf4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:26.459117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:26.459884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:26.460243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:26.461373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:26.471734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:26.473784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:26.504115 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1394/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:26 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:26.849651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:26.851138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:26.986257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eccbb3-66b7-4f7e-bf57-bce182fa1037 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:26.992260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:26.992260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:26.992260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:26.992260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] PATH_INFO: `/v3/projects/3a215e5d0f694eb28b015a77a65fed65` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:26.992764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a215e5d0f694eb28b015a77a65fed65)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:26.995124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:27.073011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:27.078596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:27.086780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec87b6-f9ee-4615-bcff-b99ea21cb673 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:27.098667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a215e5d0f694eb28b015a77a65fed65', 'request_id': 'req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.108039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:27.110054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a215e5d0f694eb28b015a77a65fed65', 'request_id': 'req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.120415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a215e5d0f694eb28b015a77a65fed65', 'request_id': 'req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.135282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] The token cache is being invalidate because project 3a215e5d0f694eb28b015a77a65fed65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:27.135662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.136456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.138273 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1444/2838] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:26 2022] DELETE /identity/v3/projects/3a215e5d0f694eb28b015a77a65fed65 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:27.142149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:27.144508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:27.231172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:27.232053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:27.232342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:27.232621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] PATH_INFO: `/v3/users/36cada5c067342218850b5d13a17627c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:27.234575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36cada5c067342218850b5d13a17627c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:27.236690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:27.286407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4091959-bbe8-4c8d-b6cd-5656f6cb364f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:27.291781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:27.292423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:27.292703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:27.293068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] PATH_INFO: `/v3/users/2a7fd61a8e8b441396226391a9e6c79d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:27.294234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7fd61a8e8b441396226391a9e6c79d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:27.296441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:27.325772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36cada5c067342218850b5d13a17627c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.327370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36cada5c067342218850b5d13a17627c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.342653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36cada5c067342218850b5d13a17627c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.363321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36cada5c067342218850b5d13a17627c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.366769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-51b640ca-ed86-45e2-9db1-d04cafceaca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36cada5c067342218850b5d13a17627c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.368466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7fd61a8e8b441396226391a9e6c79d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.369223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7fd61a8e8b441396226391a9e6c79d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.380440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7fd61a8e8b441396226391a9e6c79d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.397089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1395/2839] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:27 2022] DELETE /identity/v3/users/36cada5c067342218850b5d13a17627c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:27.402567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:27.412512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:27.413210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:27.421015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7fd61a8e8b441396226391a9e6c79d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.422128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7fd61a8e8b441396226391a9e6c79d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:27.478095 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1445/2840] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:27 2022] DELETE /identity/v3/users/2a7fd61a8e8b441396226391a9e6c79d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:27.478957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:27.481137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:27.554531 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1396/2841] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:27.583155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:27.585002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:27.732011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:27.748203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60448-d539-4d28-baf7-aa7d59462e67 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:27.759362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:27.760155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:27.760443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:27.760717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:27.785828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:27.787610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc18c2-bad9-47fb-a9ce-3c39c5e711de service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:27.819632 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1446/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:28.031202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.032911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:28.058098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa786b0-38b1-4fa2-926e-44109f5b7a71 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:28.072646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:28.073341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.073623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.073895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] PATH_INFO: `/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.075068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e829a2e0f6247a5b1b5faac26d4028c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:28.077009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:28.136738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:28.157911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e829a2e0f6247a5b1b5faac26d4028c', 'request_id': 'req-b43ac331-baa2-4134-81ee-7579c8f39317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:28.169079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e829a2e0f6247a5b1b5faac26d4028c', 'request_id': 'req-b43ac331-baa2-4134-81ee-7579c8f39317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:28.180606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e829a2e0f6247a5b1b5faac26d4028c', 'request_id': 'req-b43ac331-baa2-4134-81ee-7579c8f39317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:28.230791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] The token cache is being invalidate because project 3e829a2e0f6247a5b1b5faac26d4028c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:28.231717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:28.235211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b43ac331-baa2-4134-81ee-7579c8f39317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:28.240129 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1397/2843] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:28 2022] DELETE /identity/v3/projects/3e829a2e0f6247a5b1b5faac26d4028c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:28.245289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.246208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.246700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.287934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:28.316709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:28.535236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1447/2844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:28.543914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.546215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.546855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.601799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:28.604371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:28.622338 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1398/2845] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:28.626410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.644514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f41ad-0c32-4132-9f7e-c0109959fa01 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:28.662932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:28.663565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.663855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.664110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.664933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:28.665288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:28.667417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:28.686965 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1448/2846] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:55:28.688195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.700426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ffb8e-bf83-40e8-8318-a45faa6f93cd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:28.705285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:28.706707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.707421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.707421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.707825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:28.709761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:28.764580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1399/2847] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:28 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:28.769849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.781222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85204f-7f22-440e-9630-78e870b4a698 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:28.854130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.856132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:28.875990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:28.876717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.877069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.877369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.878774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:28.881375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:28.934311 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1449/2848] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:28 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:28.937044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:28.965217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced93dbf-18f5-4e18-b271-35ad1343a068 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:28.975941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:28.976562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:28.976839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:28.977104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:28.977901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:28.982036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.007303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1450/2849] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:29.008922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:29.024493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29177991-2848-4811-933f-0ff6946de52d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.031970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fd5ff-2ead-4d30-9ef7-0c5f68940d27 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.040167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:29.041307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.041585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.041856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.042646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:29.044663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.053286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:29.054411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.055314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.055850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] PATH_INFO: `/v3/projects/2b59c07031e64582ac08d14f9aa85a68` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.058161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b59c07031e64582ac08d14f9aa85a68)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:29.061459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.068333 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1451/2850] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:29.073005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:29.095183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b859da9-db08-4100-a89a-eebdc985f2b4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.107275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:29.108562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.110739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.111120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] PATH_INFO: `/v3/projects/ce9e67a8883f4d37a7da86febec31fa3/users/6fab725854974a6c86e089f32e446954/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.168591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9e67a8883f4d37a7da86febec31fa3, user_id=6fab725854974a6c86e089f32e446954, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:29.174049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4f84f-8b71-419a-85fd-890dd22ad2b8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.208913 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1452/2851] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:29 2022] PUT /identity/v3/projects/ce9e67a8883f4d37a7da86febec31fa3/users/6fab725854974a6c86e089f32e446954/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:29.219997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.220364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.222804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b59c07031e64582ac08d14f9aa85a68', 'request_id': 'req-d4ac191b-ba95-45f7-96c5-cbc3251bf383'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.223206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.241260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b59c07031e64582ac08d14f9aa85a68', 'request_id': 'req-d4ac191b-ba95-45f7-96c5-cbc3251bf383'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.254334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b59c07031e64582ac08d14f9aa85a68', 'request_id': 'req-d4ac191b-ba95-45f7-96c5-cbc3251bf383'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.306668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] MFA Rules not processed for user `6fab725854974a6c86e089f32e446954`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:29.309492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:29.317661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] The token cache is being invalidate because project 2b59c07031e64582ac08d14f9aa85a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:29.317990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.325420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d4ac191b-ba95-45f7-96c5-cbc3251bf383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.329580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:29.330545 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1400/2852] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:28 2022] DELETE /identity/v3/projects/2b59c07031e64582ac08d14f9aa85a68 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:29.334831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1815b27-f0ba-46cc-8328-832ee288133b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.337519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1815b27-f0ba-46cc-8328-832ee288133b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.337981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1815b27-f0ba-46cc-8328-832ee288133b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.412487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1815b27-f0ba-46cc-8328-832ee288133b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:29.413785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1815b27-f0ba-46cc-8328-832ee288133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:29.458040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:29.518176 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1453/2853] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:29.588549 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1401/2854] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:29.598573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:29.599326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.599623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.599904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.600319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:29.626127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:29.627422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:29.638263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1402/2855] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:29.639161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:29.640460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:29.642656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6139-771a-464a-bd59-484b5ee268c4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.662441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:29.663091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.663426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.663670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] PATH_INFO: `/v3/users/2d241b5c9b654fa7bf97f3652186ea6f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.665806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d241b5c9b654fa7bf97f3652186ea6f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:29.670390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.774226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac67f4f-c0bc-4a59-bacf-c33f69db7f58 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.782196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:29.782930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.783259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.783761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.796097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d241b5c9b654fa7bf97f3652186ea6f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.797079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d241b5c9b654fa7bf97f3652186ea6f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.811563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:29.816916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:29.829247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d241b5c9b654fa7bf97f3652186ea6f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.855800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1403/2856] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:29.869288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d241b5c9b654fa7bf97f3652186ea6f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.874667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ee494c8-a739-4a4e-9df9-0829895c716d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d241b5c9b654fa7bf97f3652186ea6f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:29.878912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:29.916813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1454/2857] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:29 2022] DELETE /identity/v3/users/2d241b5c9b654fa7bf97f3652186ea6f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:29.919566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:29.919874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:29.920132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:29.966578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7b519-b824-4a98-b161-37629652f154 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:29.998354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:30.000091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:30.146028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:30.146772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.147551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.149735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.151268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:30.155019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:30.157305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:30.198012 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1404/2858] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:30.203596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:30.215943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:30.233420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea8e75-4019-40db-b4ad-e0d0714b62cd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:30.244221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:30.244855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.245138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.245411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.246075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:30.248402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:30.250781 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1455/2859] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:30.279776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:30.281674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:30.327957 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1405/2860] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:30 2022] POST /identity/v3/projects => generated 397 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:30.334987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:30.377406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:30.382928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:30.437181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25dd2f8-1428-48dc-9240-a6d28b22d798 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:30.559548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:30.560425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.560811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.561191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.776246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:30.789803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52645adc-cd95-4c37-bf89-f1e7ae7ed612 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:30.802761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe65b0b-984b-411b-889c-da1382655385 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:30.806105 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1456/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:55:30.807699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.814211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.814696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.836711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:30.839737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.841912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.842692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.843942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:30.853104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:30.858864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:30.860723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:30.923665 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1406/2862] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:30 2022] POST /identity/v3/users => generated 299 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:30.927776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:30.930656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:30.955930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c79f-3ba6-418f-9ea0-41ca7a4bc697 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:30.963415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:30.964557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:30.965098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:30.965598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] PATH_INFO: `/v3/users/936372eff4da45e281fc5512632a12b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:30.967174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936372eff4da45e281fc5512632a12b6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:30.974043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:31.048683 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1457/2863] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:31.049768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:31.142635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf02de-5d72-4411-a530-dbdcc50178f9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:31.166578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:31.167215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:31.167495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:31.167769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:31.172401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936372eff4da45e281fc5512632a12b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:31.173679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936372eff4da45e281fc5512632a12b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:31.175790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:31.177446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:31.204976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936372eff4da45e281fc5512632a12b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:31.227942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1458/2864] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:31.227942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:31.227942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:31.238176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936372eff4da45e281fc5512632a12b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:31.239324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936372eff4da45e281fc5512632a12b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:31.339122 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1407/2865] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:30 2022] DELETE /identity/v3/users/936372eff4da45e281fc5512632a12b6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:31.343300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:31.345046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:31.386444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:31.612664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ac147-0f4d-4399-95c3-268ba607eb18 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:31.625082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5c4b5-78db-43e6-b261-4d144831a669 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:31.647933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:31.651706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:31.652466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:31.654224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:31.657430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:31.663683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a732a73-8366-4830-bdad-3fc4834e2866 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:31.703519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1408/2866] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:31.705802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:31.706973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:31.707322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:31.799498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:31.804901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:31.830530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1409/2867] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:31.833142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:31.834922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:31.846203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:31.880592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c27e3-3cd8-4a1e-8a4f-cec7dc89ef9e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:31.913185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:31.913705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:31.913980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:31.914281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:31.916614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:31.917564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:31.918066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:31.918587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] PATH_INFO: `/v3/projects/66a0417c2ae142c48e41a41a495ea988` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:31.919837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:31.920934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a0417c2ae142c48e41a41a495ea988)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:31.923601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:32.114212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a0417c2ae142c48e41a41a495ea988', 'request_id': 'req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:32.132804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a0417c2ae142c48e41a41a495ea988', 'request_id': 'req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:32.134369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:32.136686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:32.149021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a0417c2ae142c48e41a41a495ea988', 'request_id': 'req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:32.161659 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1459/2868] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:55:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 937 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:55:32.161659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:32.174334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6ec7-01df-4640-8652-6a3d11100296 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:32.185967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] The token cache is being invalidate because project 66a0417c2ae142c48e41a41a495ea988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:32.186360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:32.204802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:32.207972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1410/2869] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:31 2022] DELETE /identity/v3/projects/66a0417c2ae142c48e41a41a495ea988 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:32.208916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:32.211560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.334204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:32.335107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.335389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.335664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] PATH_INFO: `/v3/projects/a1d9e5ac0e5f4e8598a63c55c639ffaf/users/6453a6150dcc44bb94c257c6c7666d48/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.376388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:32.389000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d9e5ac0e5f4e8598a63c55c639ffaf, user_id=6453a6150dcc44bb94c257c6c7666d48, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:32.393877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035f78f-d994-4c6f-bed2-8bdf71785323 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:32.408726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8a8aa-6ada-464a-87ba-d9147060ffb9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:32.451530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1460/2870] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:32 2022] PUT /identity/v3/projects/a1d9e5ac0e5f4e8598a63c55c639ffaf/users/6453a6150dcc44bb94c257c6c7666d48/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:32.453461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51cd48-51e0-4291-b4f5-442ff731db5e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.453461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51cd48-51e0-4291-b4f5-442ff731db5e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.453461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51cd48-51e0-4291-b4f5-442ff731db5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.512534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1e51cd48-51e0-4291-b4f5-442ff731db5e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:32.515862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e51cd48-51e0-4291-b4f5-442ff731db5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.574203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:32.575221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.575649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.576240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.581723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.689402 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1461/2871] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:32.692343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518deb-b18d-4685-a027-788da748b965 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.692735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518deb-b18d-4685-a027-788da748b965 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.693107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d518deb-b18d-4685-a027-788da748b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.704768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:32.722069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:32.723614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa74526-db82-4008-88da-62a337dc9ba1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:32.725823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d518deb-b18d-4685-a027-788da748b965 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:32.727076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d518deb-b18d-4685-a027-788da748b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.739824 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1462/2872] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:32.740701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.741079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.741460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.742830 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1411/2873] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:32.745924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.748272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.749053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.767473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] MFA Rules not processed for user `6fab725854974a6c86e089f32e446954`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:32.769755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.818134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:32.819444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.876680 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1412/2874] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:32.879080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:32.883041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:32.931907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc96d38-c473-44a8-8df7-c54c4c048740 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:32.944512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:32.949978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:32.950559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:32.952493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:32.969652 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1463/2875] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:32.969652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:32.989475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:32.996419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.016830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1f65-4a79-47d8-bb57-de1ab81a8a6e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.047365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.048727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.049049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.049460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.050263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:33.050692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:33.051344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1413/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:33.053613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.093005 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1464/2877] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:33.097561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:33.115302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ece887-61ce-4c3f-8dfb-225a4c890084 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.127197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.128167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.130850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.131217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.132784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:33.137854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.221022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1414/2878] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:33 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:33.226158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:33.249882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b8b41-94f5-470d-8b18-ad754277436b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.504986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.507393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.508136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.508906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.533318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:33.546049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.633602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:33.638600 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1465/2879] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:33 2022] POST /identity/v3/users => generated 294 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:33.666012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe700-2641-4b5f-8eae-a6751d31f300 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.678570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.679465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.679938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.680368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.682766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:33.685189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.718780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:33.730733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1415/2880] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:33.731043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:33.742620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658840-2fef-467f-9a8d-1ed9782f4306 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.764347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.765194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.765602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.765955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.766728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:33.770821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2773df-3455-4874-af36-24fe400a62a7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:33.833378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1466/2881] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:33.839882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:33.905026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.905432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.905698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0725a7-1466-460f-a285-8151ab27244f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:33.905887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.923740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:33.928427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:33.928776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:33.929045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] PATH_INFO: `/v3/projects/bb818297b8a64e7ababc5eff3f4a1784/users/e6b8fe778b5447e4be355f476052ba5c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:33.951988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:33.954982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:34.005723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb818297b8a64e7ababc5eff3f4a1784, user_id=e6b8fe778b5447e4be355f476052ba5c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:34.035672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35469083-f2ae-4192-8887-f8cd2cf17463 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:34.093444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1416/2882] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:33 2022] PUT /identity/v3/projects/bb818297b8a64e7ababc5eff3f4a1784/users/e6b8fe778b5447e4be355f476052ba5c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:34.100671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:34.101174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:34.103022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:34.165313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1467/2883] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:34.178126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:34.179409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:34.180599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:34.181427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:34.359854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1417/2884] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:34.395187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:34.429805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ec854-e625-4352-b606-1cb8b07e4f3c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:34.542931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5a489-3d94-43ea-b342-b0bdb0c7949f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:34.551337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:34.552498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:34.553053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:34.554368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:34.569001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:34.571379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:34.591027 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1418/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:34.593516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:34.603880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:34.608891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:34.609695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:34.609987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:34.610278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:34.637907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40589f-a6d0-46db-9355-6481bc001c7e service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:34.689533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:34.690760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:34.694949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:34.695267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] PATH_INFO: `/v3/projects/73bec7c48b07450f95a29a35e113153d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:34.699500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bec7c48b07450f95a29a35e113153d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:34.712994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:34.743710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:34.758049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:34.791795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:34.793791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:34.809114 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1468/2886] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:55:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:55:34.841558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bec7c48b07450f95a29a35e113153d', 'request_id': 'req-0e362164-d82c-4b04-97d7-18f0c89de41c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:34.861170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bec7c48b07450f95a29a35e113153d', 'request_id': 'req-0e362164-d82c-4b04-97d7-18f0c89de41c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:34.892489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bec7c48b07450f95a29a35e113153d', 'request_id': 'req-0e362164-d82c-4b04-97d7-18f0c89de41c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:34.936311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] The token cache is being invalidate because project 73bec7c48b07450f95a29a35e113153d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:34.938487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:34.942656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:34.943940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1419/2887] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:34 2022] DELETE /identity/v3/projects/73bec7c48b07450f95a29a35e113153d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:34.948512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:34.965558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912deef2-991b-4bc9-98ea-832c47a4ecaf service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:35.030237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:35.031689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:35.113602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:35.114602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:35.116607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:35.117165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] PATH_INFO: `/v3/users/fb94fdc9903e477d9dfb06cb82c5e365` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:35.119200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb94fdc9903e477d9dfb06cb82c5e365)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:35.129466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:35.138942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e362164-d82c-4b04-97d7-18f0c89de41c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:35.160278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:35.161278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:35.161952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:35.162395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] PATH_INFO: `/v3/users/6fab725854974a6c86e089f32e446954` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:35.172977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fab725854974a6c86e089f32e446954)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:35.177134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:35.290567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb94fdc9903e477d9dfb06cb82c5e365'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.291570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb94fdc9903e477d9dfb06cb82c5e365'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.303576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fab725854974a6c86e089f32e446954'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.304288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb94fdc9903e477d9dfb06cb82c5e365'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.310167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fab725854974a6c86e089f32e446954'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.329883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fab725854974a6c86e089f32e446954'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.339514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb94fdc9903e477d9dfb06cb82c5e365'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.341315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44514520-bbed-4bed-9e18-ab2481d8bb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb94fdc9903e477d9dfb06cb82c5e365'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.348207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fab725854974a6c86e089f32e446954'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.351220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fab725854974a6c86e089f32e446954'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:35.400579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1469/2888] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:34 2022] DELETE /identity/v3/users/fb94fdc9903e477d9dfb06cb82c5e365 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:35.423047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1420/2889] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:35 2022] DELETE /identity/v3/users/6fab725854974a6c86e089f32e446954 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:35.425556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:35.425921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:35.426536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:35.474292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:35.481282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:35.604137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1470/2890] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:35.623806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:35.625293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:35.703983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:35.790393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f663c9-dfd7-4ae6-a861-811f4ceee809 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:35.799596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:35.801257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:35.802134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:35.803749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:35.825597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:35.828580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329547eb-c25b-4448-b9ed-7d7406830eff service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:35.847464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1421/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:36.118650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.122921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:36.139766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:36.155358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36089e-d156-4318-96ab-14343317ad2f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.166952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.167996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.168377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.168747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] PATH_INFO: `/v3/projects/05eca738faf2402faa48e15a456b8f54` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.171267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05eca738faf2402faa48e15a456b8f54)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.173252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.241931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05eca738faf2402faa48e15a456b8f54', 'request_id': 'req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:36.249235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05eca738faf2402faa48e15a456b8f54', 'request_id': 'req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:36.260541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05eca738faf2402faa48e15a456b8f54', 'request_id': 'req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:36.278195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] The token cache is being invalidate because project 05eca738faf2402faa48e15a456b8f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:36.278488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:36.279156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6daa7a99-c5fa-4afa-82b0-4d48a31595d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:36.281156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1471/2892] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:36 2022] DELETE /identity/v3/projects/05eca738faf2402faa48e15a456b8f54 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:36.286830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.293099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.293978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.351709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:36.359598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:36.485085 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1422/2893] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:36.493456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.493782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.495060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.520624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:36.522904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:36.537379 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1472/2894] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:36.540590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.553991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304dd4c-5699-4c9b-acd2-0e5b8af44ae1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.560639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.561449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.561899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.562340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.563258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:36.563931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.566732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.589571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1423/2895] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:36.590001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.604698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411f6a-9add-4d74-a954-ea220a8692ee None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.618153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.618808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.619280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.619709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.620517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.622633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.657179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1473/2896] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:36 2022] POST /identity/v3/projects => generated 393 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:36.662149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.675698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e721f9-da75-4067-8f71-3db026900543 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.786017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.800498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.800838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.801584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.804333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.806537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.841228 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1424/2897] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:36 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:36.845578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.860006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151b46c-f8dd-41ea-a459-bbb7595b71c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.873510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.875430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.875819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.876266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.877482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.879960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.896631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1474/2898] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:36.900952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.912458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27320f7a-b4ba-4de1-98b8-9ace10fde9f1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:36.926916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:36.927680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:36.927962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:36.928309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:36.929123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:36.931121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd38ff-ecbf-42a7-a4fa-b94df04b6f6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:36.960000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1425/2899] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:36.964188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:36.984252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ba0df-ef67-4a1c-bf63-ac2d77166aea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:37.000985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:37.001688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:37.002086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:37.002437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] PATH_INFO: `/v3/projects/d982e19aac5c41c0846e2e7b71c6d75a/users/b4f97d7719ae4acfb3e4c20a59cdc7ab/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:37.040745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d982e19aac5c41c0846e2e7b71c6d75a, user_id=b4f97d7719ae4acfb3e4c20a59cdc7ab, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:37.046058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:37.074521 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1475/2900] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:36 2022] PUT /identity/v3/projects/d982e19aac5c41c0846e2e7b71c6d75a/users/b4f97d7719ae4acfb3e4c20a59cdc7ab/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:37.084973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:37.085591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:37.085872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:37.135735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] MFA Rules not processed for user `b4f97d7719ae4acfb3e4c20a59cdc7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:37.141668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:37.171773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:37.282659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1426/2901] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:37.289161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:37.476289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4d5a1-56c4-47b8-92a3-7cc67d3181f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:37.485245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:37.487207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:37.487608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:37.488984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:37.489947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:37.499326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:37.569694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:37.571494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1476/2902] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:37 2022] POST /identity/v3/projects => generated 393 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:37.613164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7113e4-2a62-43c0-b17c-844cc21f7790 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:37.666105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:37.669422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:37.695572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:37.888870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:37.889495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:37.889776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:37.894770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:37.895787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:37.899262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:37.926313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13403fe-d726-4d65-b161-e698f47d753c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:37.932230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:37.933004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:37.933290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:37.933567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] PATH_INFO: `/v3/projects/ce9e67a8883f4d37a7da86febec31fa3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:37.934885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9e67a8883f4d37a7da86febec31fa3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:37.937677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:37.963882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1427/2903] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:37 2022] POST /identity/v3/users => generated 303 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:37.967820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:37.992241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb130ae-b072-42f2-aaa8-e60438fb82a3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.001170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.001787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.002061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.002330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.003577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.006938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.032831 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1428/2904] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:38.034805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:38.056062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6388d6-317d-499a-997d-e1894460ea90 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.058948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9e67a8883f4d37a7da86febec31fa3', 'request_id': 'req-d37ae8f7-1517-4b6f-b136-5344064196df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:38.071832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9e67a8883f4d37a7da86febec31fa3', 'request_id': 'req-d37ae8f7-1517-4b6f-b136-5344064196df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:38.082202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9e67a8883f4d37a7da86febec31fa3', 'request_id': 'req-d37ae8f7-1517-4b6f-b136-5344064196df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:38.103245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] The token cache is being invalidate because project ce9e67a8883f4d37a7da86febec31fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:38.103855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:38.104784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d37ae8f7-1517-4b6f-b136-5344064196df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:38.111343 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1477/2905] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:37 2022] DELETE /identity/v3/projects/ce9e67a8883f4d37a7da86febec31fa3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:38.114276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.114807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.115294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.146464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:38.148157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:38.222457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.223490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.223930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.231230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] PATH_INFO: `/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9/users/9e0531aad05d44c19884d3f12972e8c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.267008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c0d956ef0248bebbdb8e679d65b2b9, user_id=9e0531aad05d44c19884d3f12972e8c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.274585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab77ff5-f38c-4859-b357-3be53939acda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.305511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1478/2906] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:38.309451 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1429/2907] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:38 2022] PUT /identity/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9/users/9e0531aad05d44c19884d3f12972e8c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:38.323595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:38.325109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.325589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.326027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.333252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:38.352582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce89ce-0e86-47d5-b0fa-c99598919ec8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.384829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:38.388235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:38.427095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:38.428269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:38.494766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.495757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.502919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.503342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.504039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.510333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.516254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1430/2908] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:38.522462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:38.528945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1479/2909] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:38.537224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:38.545613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208537d1-1670-410a-93d6-fe36e7654b23 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.552665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0347-4144-4cde-85ea-fba221ac2f67 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.557977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.558664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.558840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.558977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.559262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.559782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.559913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:38.560212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.560342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.560656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] PATH_INFO: `/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9/users/9e0531aad05d44c19884d3f12972e8c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.563389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c0d956ef0248bebbdb8e679d65b2b9, user_id=9e0531aad05d44c19884d3f12972e8c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.567815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d8c7-98dc-4abb-9ffd-e9ba5d2bd2ae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.580428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.597389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1480/2910] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:38 2022] PUT /identity/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9/users/9e0531aad05d44c19884d3f12972e8c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:38.599392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.599798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.600168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.628319 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1431/2911] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:38.635113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:38.668599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f6f8b-9466-4c89-a87d-721b7b73e5a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:38.671807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] MFA Rules not processed for user `9e0531aad05d44c19884d3f12972e8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:38.673407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:38.874293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:38.874939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:38.875216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:38.875819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:38.876420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:38.880183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:38.968720 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1481/2912] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:38.989167 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1432/2913] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:38 2022] POST /identity/v3/projects => generated 375 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:38.993353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.009724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bea054-7f19-4d42-85df-425afa6049b6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.070771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.073057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:39.143956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.144839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.145190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.145529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.146243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.151002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.205214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1482/2914] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:38 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:39.206734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70efce-9ba7-48e9-9ae0-845422c43e6d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.210365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.214006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.214731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.214995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.215277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.235258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773438e-07c7-41d8-ba6e-71c912e602d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.249436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.250055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.250334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.250635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.251294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.252717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:39.253062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.281989 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1483/2915] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:39.292429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.308244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75909894-b1e5-4643-ac97-ce848688af30 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.312813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.315988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.321048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.322798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.323709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.324634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.326190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.326964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1433/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:39.332751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630076dc-cc09-4191-aa8b-df4b9da450e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.370779 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1484/2917] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:55:39.372824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.395687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e02ac-2803-413f-a49b-b24520acc26a service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.403905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.404876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.405471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.405905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] PATH_INFO: `/v3/projects/9b6e724ba5a849a2ac33fba2b3868ccc/users/6c381100cbcb4f128438e334e9beca6b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.486626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6e724ba5a849a2ac33fba2b3868ccc, user_id=6c381100cbcb4f128438e334e9beca6b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.502291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.527007 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1434/2918] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:39 2022] PUT /identity/v3/projects/9b6e724ba5a849a2ac33fba2b3868ccc/users/6c381100cbcb4f128438e334e9beca6b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:39.531900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.532785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.532785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.560760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:39.563648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:39.577565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:39.688775 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1485/2919] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:39.707093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:39.708879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:39.788043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2f6e8-e2a8-4418-8f2e-d53c801908c9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:39.792402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:39.793132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:39.793535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:39.793910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:39.807431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:39.809002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:39.823735 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1435/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:40.304057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:40.438999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19545a44-bbf0-4e40-af20-eeac7bded98b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:40.447905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:40.448805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:40.449096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:40.449413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:40.452035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:40.537716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:40.543906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe7842-010d-42f3-9569-47bdfd51df1a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:40.567911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1486/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:40.587263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:40.597933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a3d4-18b4-4abd-8ea8-eb0183ce566f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:40.632329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:40.633074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:40.633416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:40.633771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] PATH_INFO: `/v3/users/b4f97d7719ae4acfb3e4c20a59cdc7ab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:40.638468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f97d7719ae4acfb3e4c20a59cdc7ab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:40.644604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:40.774143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f97d7719ae4acfb3e4c20a59cdc7ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:40.777095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f97d7719ae4acfb3e4c20a59cdc7ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:40.797322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f97d7719ae4acfb3e4c20a59cdc7ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:40.815465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f97d7719ae4acfb3e4c20a59cdc7ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:40.827834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f97d7719ae4acfb3e4c20a59cdc7ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:40.888900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1436/2922] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:40 2022] DELETE /identity/v3/users/b4f97d7719ae4acfb3e4c20a59cdc7ab => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:40.904012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:40.904261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:40.904602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:40.930720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:40.932128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:41.044359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:41.115149 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1487/2923] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:41.190970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:41.193135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:41.236410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:41.334278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc75d-53db-4267-81d9-365147beddae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:41.338794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:41.339411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:41.339687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:41.339957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:41.352907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:41.355568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:41.375416 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1437/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:44.757142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:44.763038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:44.807106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf44e7-1aa0-4228-b94e-43c9869560eb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:44.821866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:44.823975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:44.825101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:44.825827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] PATH_INFO: `/v3/projects/d982e19aac5c41c0846e2e7b71c6d75a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:44.838380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d982e19aac5c41c0846e2e7b71c6d75a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:44.845267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:44.955087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd982e19aac5c41c0846e2e7b71c6d75a', 'request_id': 'req-5489b0bf-8169-4d60-89de-a5b5f3463e2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:44.990063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd982e19aac5c41c0846e2e7b71c6d75a', 'request_id': 'req-5489b0bf-8169-4d60-89de-a5b5f3463e2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.010216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd982e19aac5c41c0846e2e7b71c6d75a', 'request_id': 'req-5489b0bf-8169-4d60-89de-a5b5f3463e2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.043309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] The token cache is being invalidate because project d982e19aac5c41c0846e2e7b71c6d75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:45.043872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.052153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.055158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1488/2925] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:44 2022] DELETE /identity/v3/projects/d982e19aac5c41c0846e2e7b71c6d75a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:45.059872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:45.080135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb1ccd-1ab9-40a4-a3bd-e798a7439885 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:45.249813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:45.253700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:45.253700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:45.253700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] PATH_INFO: `/v3/users/9e0531aad05d44c19884d3f12972e8c1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:45.258288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0531aad05d44c19884d3f12972e8c1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:45.262968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:45.396816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0531aad05d44c19884d3f12972e8c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.397884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0531aad05d44c19884d3f12972e8c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.416447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0531aad05d44c19884d3f12972e8c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.432389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0531aad05d44c19884d3f12972e8c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.438934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0531aad05d44c19884d3f12972e8c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:45.509925 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1438/2926] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:45 2022] DELETE /identity/v3/users/9e0531aad05d44c19884d3f12972e8c1 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:46.633030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:46.634745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:46.833948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489b0bf-8169-4d60-89de-a5b5f3463e2d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:46.991136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:46.992844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:46.992844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:46.992844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:46.995431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:47.019429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:47.120098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:47.122079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb72b24-d0c4-4ce8-a2db-97fff876e5f2 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:47.141637 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1489/2927] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:48.155850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:48.157423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:48.261025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:48.322133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94dfd3-c2f5-456a-a0d3-b6ffb54b6cf6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:48.344310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:48.345258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:48.345736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:48.346229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] PATH_INFO: `/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:48.348315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c0d956ef0248bebbdb8e679d65b2b9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:48.350526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:48.439420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c0d956ef0248bebbdb8e679d65b2b9', 'request_id': 'req-5e54b14f-1be6-49fa-923b-421f420d35a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:48.470342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c0d956ef0248bebbdb8e679d65b2b9', 'request_id': 'req-5e54b14f-1be6-49fa-923b-421f420d35a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:48.484086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c0d956ef0248bebbdb8e679d65b2b9', 'request_id': 'req-5e54b14f-1be6-49fa-923b-421f420d35a1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:48.523009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] The token cache is being invalidate because project 91c0d956ef0248bebbdb8e679d65b2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:48.523354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:48.524040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e54b14f-1be6-49fa-923b-421f420d35a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:48.526360 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1439/2928] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:48 2022] DELETE /identity/v3/projects/91c0d956ef0248bebbdb8e679d65b2b9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:48.532465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:48.532988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:48.533530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:48.572794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:48.587518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:48.700567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1490/2929] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:48.710998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:48.711966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:48.712452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:48.740599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:48.742253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:48.752598 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1440/2930] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:48.756822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:48.772347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58738cb6-a0d3-4624-bdfd-6b29cde3cad7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:48.779096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:48.780167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:48.780668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:48.781173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:48.782078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:48.782655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:48.786740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:48.807026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1491/2931] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:48.811410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:48.833866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f94ad-c468-4da3-8c9a-9cb0db0d9bfe None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:48.841112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:48.841887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:48.842263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:48.842800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:48.843498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:48.846105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:48.893016 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1441/2932] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:48 2022] POST /identity/v3/projects => generated 413 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:48.900921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:48.934148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec5a12-cc6b-46d6-b1f2-bb64253c83ce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:49.069735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:49.070548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.070940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.071309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.072160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:49.074391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:49.122277 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1492/2933] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:48 2022] POST /identity/v3/users => generated 307 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:49.127898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:49.149124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451155a3-09cf-4e1a-b8d2-cd4036e83f87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:49.157318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:49.161786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.162275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.162741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.163677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:49.166313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:49.180067 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1442/2934] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:49.184474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:49.202963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b7ef2-16ea-45b0-800a-3a08ec32ba91 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:49.216637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:49.217187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.217736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.218079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.219026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:49.220754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7529dc2-97d3-45b8-9d06-f1164ecd73bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:49.238911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1493/2935] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:49.242807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:49.258673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43d41-dbcd-4e19-a909-2cd2b2c9f044 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:49.271258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:49.280389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.280961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.281237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] PATH_INFO: `/v3/projects/8afe75ffdb814e0ba23546cf3866ceac/users/302ac6f497cf4d3c8c638f5b4b49f723/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.332264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afe75ffdb814e0ba23546cf3866ceac, user_id=302ac6f497cf4d3c8c638f5b4b49f723, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:49.337424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:49.359972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1443/2936] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:49 2022] PUT /identity/v3/projects/8afe75ffdb814e0ba23546cf3866ceac/users/302ac6f497cf4d3c8c638f5b4b49f723/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:49.369303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f2fe-febf-4f10-8647-779a357fcbf6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.369303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f2fe-febf-4f10-8647-779a357fcbf6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.369303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053f2fe-febf-4f10-8647-779a357fcbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.401737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5053f2fe-febf-4f10-8647-779a357fcbf6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:49.403521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5053f2fe-febf-4f10-8647-779a357fcbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:49.417147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:49.499105 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1494/2937] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:49.562005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:49.572606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:49.778390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a6b09-7d4a-4a39-9f7f-8eb7f1f7918c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:49.788552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:49.790224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:49.791169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:49.791543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:49.819887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:49.821576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:49.859605 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1444/2938] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:50.391163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393602-2182-4388-8ae7-5c0f1a57887c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:50.391627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393602-2182-4388-8ae7-5c0f1a57887c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:50.391742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393602-2182-4388-8ae7-5c0f1a57887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:50.449103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc393602-2182-4388-8ae7-5c0f1a57887c None None] MFA Rules not processed for user `e6b8fe778b5447e4be355f476052ba5c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:50.453036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc393602-2182-4388-8ae7-5c0f1a57887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:50.619335 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1495/2939] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:50.637650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:50.639511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:50.672764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:50.820000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba4cc2-245b-43c3-82b3-be59ebc4ccdc service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:50.827602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:50.828681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:50.828883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:50.829316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:50.854160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:50.859162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:50.883217 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1445/2940] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:53.379380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:53.380400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:53.381104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:53.498974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] MFA Rules not processed for user `6453a6150dcc44bb94c257c6c7666d48`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:53.502677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:53.758014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1496/2941] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:54.113145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:54.138716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d189827-3f66-46e3-901b-fdc0f2582ebf service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:54.146832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:54.148963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:54.149597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:54.152133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:54.171645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:54.175575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:54.200472 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1446/2942] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:54 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:54.600190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:54.602929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:54.694192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:54.787683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf5c6b-2140-4396-97ea-b4605f8a77b1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:54.804380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:54.805231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:54.805659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:54.806096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:54.808059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:54.834983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:54.840384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485c8f5-fbf2-4f1c-b263-115164045332 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:54.869722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1497/2943] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:55.039899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:55.042540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:55.082309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612c099-cdc4-4500-8c92-147a030483e8 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:55.097863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:55.098526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:55.098820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:55.100751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] PATH_INFO: `/v3/users/dab2b6bf40594154a6340d5afd02da0c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:55.102973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab2b6bf40594154a6340d5afd02da0c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:55.113022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:55.252834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab2b6bf40594154a6340d5afd02da0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:55.254044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab2b6bf40594154a6340d5afd02da0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:55.272207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab2b6bf40594154a6340d5afd02da0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:55.307773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab2b6bf40594154a6340d5afd02da0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:55.310954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab2b6bf40594154a6340d5afd02da0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:55.352737 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1447/2944] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:55:55 2022] DELETE /identity/v3/users/dab2b6bf40594154a6340d5afd02da0c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:55.405276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:55.406414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:55.406893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:55.449615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] MFA Rules not processed for user `6c381100cbcb4f128438e334e9beca6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:55.451226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:55.588208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1498/2945] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:55.917669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:55.919415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:56.052204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:56.114064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a3c3a-106f-405b-a035-f65c08ec5710 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:56.115238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:56.118676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:56.137968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:56.142243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.144662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.145235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] PATH_INFO: `/v3/projects/8dc5dc16f54e4d2991a2b456216459a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.153982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc5dc16f54e4d2991a2b456216459a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:56.162851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:56.290232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc5dc16f54e4d2991a2b456216459a4', 'request_id': 'req-765f8130-1247-4aa5-805a-695c0a595229'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:56.320508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc5dc16f54e4d2991a2b456216459a4', 'request_id': 'req-765f8130-1247-4aa5-805a-695c0a595229'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:56.320883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410c51-4542-457e-873d-236cf2638dd6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:56.334041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc5dc16f54e4d2991a2b456216459a4', 'request_id': 'req-765f8130-1247-4aa5-805a-695c0a595229'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:56.350784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] The token cache is being invalidate because project 8dc5dc16f54e4d2991a2b456216459a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:55:56.351336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:56.352985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-765f8130-1247-4aa5-805a-695c0a595229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:55:56.360088 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1448/2946] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:55:55 2022] DELETE /identity/v3/projects/8dc5dc16f54e4d2991a2b456216459a4 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:55:56.365704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff426b-577c-4b6b-81ca-61ded9263bca None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.366309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff426b-577c-4b6b-81ca-61ded9263bca None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.366833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff426b-577c-4b6b-81ca-61ded9263bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.410983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f3ff426b-577c-4b6b-81ca-61ded9263bca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:56.412652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3ff426b-577c-4b6b-81ca-61ded9263bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:56.441529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:56.482808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:56.484803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.485629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.486870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.490340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:56.554864 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1449/2947] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:56.562754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.563034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.563307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.591371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:56.593556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:56.603011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:56.604426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:56.620936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1499/2948] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:55:56.624493 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1450/2949] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:55:56.630553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:56.656980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43007bb3-7d26-4d74-ba7a-6d36a2b71f68 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:56.663806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:56.664481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.665521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.665908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.666727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:55:56.667307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:56.669163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:56.687695 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1500/2950] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:55:56.688708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:56.703329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691616d5-1dc8-406f-86d7-3200924c99ec None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:56.717142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:56.718079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:56.718582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:56.719014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:56.719815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:56.722639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:56.832668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1451/2951] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:55:56 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:55:56.836502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:56.853644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791af5f9-a4f8-4e2c-8130-2b50f63de1bf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:57.009489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:57.011062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.011746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.012899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.013603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:57.019939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:57.090552 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1501/2952] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:55:56 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:55:57.091736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:57.119455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56909e-05b5-4519-ae68-2b10733cc262 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:57.148070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:57.151368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.152361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.168897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.170064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:57.171996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:57.216076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1452/2953] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:57.220540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:57.234154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be0c7a-2ba6-4dff-9e1e-2525e37d4a8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:57.269879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:57.271104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.271547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.272041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.272864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:57.274735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fef15-53e2-47fe-9943-e5ba347f20d9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:57.328899 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1502/2954] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:55:57.329137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:57.342929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd5776-2c3e-42a0-9dbe-430de89f3a79 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:57.353158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:57.354053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.354593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.355035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] PATH_INFO: `/v3/projects/71f6b606beca4eb5b62c89a945be15c7/users/3aa253ded2304ac3917a1788b6a1500b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.391700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f6b606beca4eb5b62c89a945be15c7, user_id=3aa253ded2304ac3917a1788b6a1500b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:57.394952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:57.426588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1453/2955] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:55:57 2022] PUT /identity/v3/projects/71f6b606beca4eb5b62c89a945be15c7/users/3aa253ded2304ac3917a1788b6a1500b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:55:57.432239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595196d-282f-4ac5-b28b-f1c625cd280d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.432993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595196d-282f-4ac5-b28b-f1c625cd280d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.433477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595196d-282f-4ac5-b28b-f1c625cd280d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.486027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4595196d-282f-4ac5-b28b-f1c625cd280d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:55:57.489172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4595196d-282f-4ac5-b28b-f1c625cd280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:57.598226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:57.645111 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1503/2956] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:55:57.656782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:55:57.658644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:55:57.694540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:55:57.797930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa5b44-3843-480a-b4c9-d9467822ab84 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:55:57.805089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:55:57.805973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:55:57.806583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:55:57.807020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:55:57.819118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:55:57.823152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053ddcb-8079-45a0-8a4f-abcda1bad9d0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:55:57.849501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1454/2957] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:03.733086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:03.733855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:03.733855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:03.793639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] MFA Rules not processed for user `302ac6f497cf4d3c8c638f5b4b49f723`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:03.795185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:03.986054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1504/2958] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:03.986542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:03.987093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:03.987213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:04.053406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] MFA Rules not processed for user `302ac6f497cf4d3c8c638f5b4b49f723`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:04.054693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:04.085121 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1455/2959] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:56:05.170300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:05.171862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:05.347285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431a5ea-1ecf-415f-a690-061ce68f9185 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:05.354812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:05.356135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:05.356654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:05.357181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:05.384195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:05.386344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:05.401507 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1505/2960] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:05 2022] GET /identity/v3/auth/tokens => generated 3290 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:06.167662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:06.169356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:06.192044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f626f7-d277-49f8-8974-3dd58fe73f66 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:06.203155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:06.204798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:06.205276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:06.205769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:06.217959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:06.234045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:06.235948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7de7d9-b7ed-4b1c-b305-9bd25354efa5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:06.258528 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1456/2961] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:08.715246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:08.718734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:08.759003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe29813-6a18-437b-a7a2-62aed9c3f0fc service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:08.780911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:08.781623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:08.781806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:08.787914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] PATH_INFO: `/v3/users/93982876b1e24efb8ffa8c009532343a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:08.790590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93982876b1e24efb8ffa8c009532343a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:08.795000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:08.915939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93982876b1e24efb8ffa8c009532343a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:08.916808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93982876b1e24efb8ffa8c009532343a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:08.936577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93982876b1e24efb8ffa8c009532343a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:08.958780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:08.965268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93982876b1e24efb8ffa8c009532343a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:08.969969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93982876b1e24efb8ffa8c009532343a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:09.012268 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1506/2962] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:08 2022] DELETE /identity/v3/users/93982876b1e24efb8ffa8c009532343a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:09.017329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:09.017805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:09.017895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:09.076874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:09.083473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:09.162337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:09.164000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:09.216907 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1457/2963] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:09.235094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:09.239911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:09.327990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719713d-1394-42ca-9f14-a9fe65fd3c24 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:09.343123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:09.345222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:09.345852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:09.348153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:09.353136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:09.414904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b500a3-e70c-46ea-add5-af7eb8769d37 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:09.420938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:09.421904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:09.424373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:09.425909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:09.463873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:09.465942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:09.509571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1458/2964] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:09.551788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:09.553547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:09.582842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1507/2965] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:10.139196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:10.141515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:10.148127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:10.150670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:10.165845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac7d14-e2f3-4f6c-87ee-42838ff6f7e3 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:10.180974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:10.182125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:10.182413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:10.182707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] PATH_INFO: `/v3/projects/349d3aa8627644d59867a744695f2579` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:10.183247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618b7f2-54a4-484e-9de6-1d5de19b9ef9 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:10.184446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349d3aa8627644d59867a744695f2579)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:10.189912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:10.191606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:10.192365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:10.192750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:10.193144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] PATH_INFO: `/v3/users/6453a6150dcc44bb94c257c6c7666d48` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:10.194743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6453a6150dcc44bb94c257c6c7666d48)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:10.199502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:10.325598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:10.330687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6453a6150dcc44bb94c257c6c7666d48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.334286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6453a6150dcc44bb94c257c6c7666d48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.347171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349d3aa8627644d59867a744695f2579', 'request_id': 'req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.366605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349d3aa8627644d59867a744695f2579', 'request_id': 'req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.367834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6453a6150dcc44bb94c257c6c7666d48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.384890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349d3aa8627644d59867a744695f2579', 'request_id': 'req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.393980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:10.397105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6453a6150dcc44bb94c257c6c7666d48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.398143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6453a6150dcc44bb94c257c6c7666d48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.411785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] The token cache is being invalidate because project 349d3aa8627644d59867a744695f2579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:10.412089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.414577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.421406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1459/2966] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:10 2022] DELETE /identity/v3/projects/349d3aa8627644d59867a744695f2579 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:10.423666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:10.424396 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1508/2967] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:10 2022] DELETE /identity/v3/users/6453a6150dcc44bb94c257c6c7666d48 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:10.427901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:10.555443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b1db1-fe1d-40de-ba64-bf0b6ea0ddb2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:10.653158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:10.654255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:10.654726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:10.655047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] PATH_INFO: `/v3/users/4349ca208839405199602d34e1c7bf33` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:10.656394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4349ca208839405199602d34e1c7bf33)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:10.659584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:10.745393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4349ca208839405199602d34e1c7bf33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.746891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4349ca208839405199602d34e1c7bf33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.762282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4349ca208839405199602d34e1c7bf33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.793388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4349ca208839405199602d34e1c7bf33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.794529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4349ca208839405199602d34e1c7bf33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:10.816498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:10.828329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1460/2968] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:10 2022] DELETE /identity/v3/users/4349ca208839405199602d34e1c7bf33 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:11.332790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:11.334539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:11.356968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796015-d329-4eba-9aeb-546db1849103 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:11.400626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:11.403167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:11.521231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:11.522860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:11.523329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:11.523626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] PATH_INFO: `/v3/projects/a1d9e5ac0e5f4e8598a63c55c639ffaf` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:11.524849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d9e5ac0e5f4e8598a63c55c639ffaf)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:11.527211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:11.592297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25e5ed-ab86-4a76-b315-b9f0f57eef37 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:11.617861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d9e5ac0e5f4e8598a63c55c639ffaf', 'request_id': 'req-f6a1df30-ed99-43f7-a52f-68216e766c29'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:11.625058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d9e5ac0e5f4e8598a63c55c639ffaf', 'request_id': 'req-f6a1df30-ed99-43f7-a52f-68216e766c29'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:11.636710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d9e5ac0e5f4e8598a63c55c639ffaf', 'request_id': 'req-f6a1df30-ed99-43f7-a52f-68216e766c29'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:11.668440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] The token cache is being invalidate because project a1d9e5ac0e5f4e8598a63c55c639ffaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:11.669337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:11.672702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f6a1df30-ed99-43f7-a52f-68216e766c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:11.676407 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1509/2969] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:11 2022] DELETE /identity/v3/projects/a1d9e5ac0e5f4e8598a63c55c639ffaf => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:11.684120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e96dfe-b9a5-4160-927e-944294b2bef0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:11.684454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e96dfe-b9a5-4160-927e-944294b2bef0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:11.684729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e96dfe-b9a5-4160-927e-944294b2bef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:11.750411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-82e96dfe-b9a5-4160-927e-944294b2bef0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:11.756694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82e96dfe-b9a5-4160-927e-944294b2bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:11.836195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:11.840978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:11.841516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:11.841655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] PATH_INFO: `/v3/projects/00041f889f3d430d87831f82a4d3b598` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:11.844682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00041f889f3d430d87831f82a4d3b598)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:11.851149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.039529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1510/2970] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:12.047643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.047930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.048421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.051727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00041f889f3d430d87831f82a4d3b598', 'request_id': 'req-21beb15e-c7fe-4289-a26c-06711798b313'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:12.074796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00041f889f3d430d87831f82a4d3b598', 'request_id': 'req-21beb15e-c7fe-4289-a26c-06711798b313'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:12.097976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00041f889f3d430d87831f82a4d3b598', 'request_id': 'req-21beb15e-c7fe-4289-a26c-06711798b313'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:12.111360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:12.116376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:12.124830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] The token cache is being invalidate because project 00041f889f3d430d87831f82a4d3b598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:12.127075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:12.129327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21beb15e-c7fe-4289-a26c-06711798b313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:12.136712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1461/2971] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:11 2022] DELETE /identity/v3/projects/00041f889f3d430d87831f82a4d3b598 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:56:12.138698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988ae7-cf10-4da9-9c64-11198a4172c7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.138985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988ae7-cf10-4da9-9c64-11198a4172c7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.139331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28988ae7-cf10-4da9-9c64-11198a4172c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.152221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:12.194048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-28988ae7-cf10-4da9-9c64-11198a4172c7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:12.196724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-28988ae7-cf10-4da9-9c64-11198a4172c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:12.260671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:12.278302 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1511/2972] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:56:12.279654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.307255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45a6e4-545f-4b0e-9635-ddefa6f63568 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.318778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.319750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.320226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.320689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.321539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:12.322048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.323942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.343496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1462/2973] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:12.345729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.346019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.346293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.379506 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1512/2974] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:56:12.382346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.407797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13f88d-40cf-4043-bd77-3df90241f372 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.409927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:12.411490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:12.422172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.425208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.426716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.427366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.428367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.436476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1463/2975] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:12.439199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.441982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.468048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb73d-06ea-4eea-8ece-439a34a9a66f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.484316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.485531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.485816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.486146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.486828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:12.487739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.489850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.514891 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1464/2976] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:56:12.515139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1513/2977] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:12.519609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.519799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.533598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b88cc-b9fe-4171-a388-a631d099871a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.658785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.658785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.658785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.658785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.659971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.659971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.662778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746ccf-3563-4379-8e0b-462008f2dd64 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.673220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.674191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.674325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.675376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.676128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.678744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.730820 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1514/2978] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:12.734666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.743212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1465/2979] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:12.746686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.759476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18f050-d42f-4116-abff-8033f43c232c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.877776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.878754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.879360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.879643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.880296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.883057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.900939 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1515/2980] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:12.902219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:12.904973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93856721-fd75-49b4-bd98-085ef7bcdf60 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.914658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab0dd-f6e2-4c3f-9cad-982dab1908b1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:12.919806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.920448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.920726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.921002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.923536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.925561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:12.927847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:12.927847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:12.928146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:12.929620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:12.945979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.949109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:12.959207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:12.974989 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1516/2981] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:12.976168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.005377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d3c7d-8b44-4ba0-aa83-1c268dc40d5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.014641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.015603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.016067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.016349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] PATH_INFO: `/v3/projects/cfa59dfe189849238f3693696fd1089e/users/76b517a69e1f4dd0895104befebc128d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.051748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa59dfe189849238f3693696fd1089e, user_id=76b517a69e1f4dd0895104befebc128d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.055002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9830ef-9ea2-413f-9b79-a7a62435c48f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.077232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1466/2982] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:12 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:13.079401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.095139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1517/2983] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:12 2022] PUT /identity/v3/projects/cfa59dfe189849238f3693696fd1089e/users/76b517a69e1f4dd0895104befebc128d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:13.097456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.098344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.098833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.121149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e61f41-c238-41b4-afe9-27fc4cb1982b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.121715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:13.123102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:13.177378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:13.230743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.231816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.232336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.233368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.235004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.236912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.253232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1467/2984] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:13.258675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.280499 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1518/2985] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:13.293238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228878cb-b51a-48aa-b71c-c951d7337a1c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.296491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.299183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:13.307023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.308014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.308944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.309599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.310800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.317075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.337643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1468/2986] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:13.339156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.351411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56b5a2-e06e-4bae-839b-624a8d1b1534 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.357127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.358201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.358788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.359110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] PATH_INFO: `/v3/projects/7aea38006c4548c48e56248dae811780/users/99525fec7f964953914363f342fc6f7f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.388279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aea38006c4548c48e56248dae811780, user_id=99525fec7f964953914363f342fc6f7f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.391667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f250c-d7b1-4fac-a92f-4dd5ca40bfce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.393380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ee44c-8112-430b-9e23-d54376c18b0d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.399187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.399995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.400285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.400684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.413052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.414578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.416665 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1469/2987] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:13 2022] PUT /identity/v3/projects/7aea38006c4548c48e56248dae811780/users/99525fec7f964953914363f342fc6f7f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:13.426918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1519/2988] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:56:13.427913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.428276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.428627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.462135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] MFA Rules not processed for user `99525fec7f964953914363f342fc6f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:13.464009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:13.547636 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1470/2989] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:13.553005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.564434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd054ea-2a50-4feb-a153-b233d22be664 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.647445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.648302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.648741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.649110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.649847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.652627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e71e-5375-46a4-82a8-60f63bdc6f2e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.687887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1520/2990] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:13 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:13.689419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.702104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53c59-8790-4a84-bcd5-6f34c405ce24 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.758942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.759227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.759527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.779213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:13.812090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.814391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.814701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.814975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.816172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.818167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.832355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] MFA Rules not processed for user `3aa253ded2304ac3917a1788b6a1500b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:13.834090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:13.877098 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1471/2991] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:13 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:13.882534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:13.909492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eea65-a7a6-4459-ab93-74d08995847a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:13.930792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:13.931578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:13.931894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:13.932259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:13.933310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:13.936529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:13.938835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:13.952427 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1472/2992] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:13.957156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:14.006618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6869e98-1187-4e21-a231-d93ac1ec9da9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:14.025300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:14.026061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:14.026684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:14.027023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] PATH_INFO: `/v3/projects/c48e668375fe4fc89ff46adafece5fd9/users/152d1f77bdbb42078b8018ae96d44b61/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:14.030921 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1521/2993] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:14.069683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48e668375fe4fc89ff46adafece5fd9, user_id=152d1f77bdbb42078b8018ae96d44b61, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:14.083546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:14.110545 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1473/2994] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:13 2022] PUT /identity/v3/projects/c48e668375fe4fc89ff46adafece5fd9/users/152d1f77bdbb42078b8018ae96d44b61/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:14.115913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:14.133369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257e225-d254-4463-9926-4f99cb5022f9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:14.216696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:14.217656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:14.218449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:14.219404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:14.220817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:14.222824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79d985-1ec7-4506-ae58-95f697a321ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:14.240817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1522/2995] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:14.252277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:14.262262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9f706-bcbb-4712-ae4c-ff6d4667ba54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:14.267220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:14.267886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:14.268163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:14.268525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] PATH_INFO: `/v3/projects/c48e668375fe4fc89ff46adafece5fd9/users/152d1f77bdbb42078b8018ae96d44b61/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:14.271141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48e668375fe4fc89ff46adafece5fd9, user_id=152d1f77bdbb42078b8018ae96d44b61, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:14.273983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:14.302631 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1474/2996] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:14 2022] PUT /identity/v3/projects/c48e668375fe4fc89ff46adafece5fd9/users/152d1f77bdbb42078b8018ae96d44b61/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:14.307120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:14.307425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:14.307677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:14.347604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] MFA Rules not processed for user `152d1f77bdbb42078b8018ae96d44b61`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:14.349345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:14.465148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:14.468826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:14.551447 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1523/2997] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:14.634272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5caaa4-b754-4d28-bc74-784b83a389b9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:14.650671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:14.651923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:14.653623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:14.654006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:14.843942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:14.846086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:14.858096 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1475/2998] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:16.022464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:16.024239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:16.069111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:16.199563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c3afa-6fb5-444e-9e1b-7b4c30bad9e2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:16.206506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:16.207150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:16.208344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:16.210094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:16.213905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:16.234396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:16.238945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:16.270730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1524/2999] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:16.308761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:16.310304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:16.406525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564db8c0-4a4e-4e41-963c-a8779350f1c6 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:16.430770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:16.431393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:16.431668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:16.431938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:16.457396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:16.464782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf45a51-6b03-4b5b-89c0-017bd201fb07 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:16.502344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1476/3000] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:19.351180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:19.542539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433d772-54ab-4e05-9310-edb3f36580a6 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:19.556811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:19.557570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:19.557868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:19.558124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] PATH_INFO: `/v3/users/99525fec7f964953914363f342fc6f7f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:19.563326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99525fec7f964953914363f342fc6f7f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:19.569063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:19.701175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99525fec7f964953914363f342fc6f7f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:19.702554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99525fec7f964953914363f342fc6f7f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:19.726056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99525fec7f964953914363f342fc6f7f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:19.749174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99525fec7f964953914363f342fc6f7f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:19.750175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99525fec7f964953914363f342fc6f7f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:19.768458 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1525/3001] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:19 2022] DELETE /identity/v3/users/99525fec7f964953914363f342fc6f7f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:56:19.771612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2002cf-71f1-4965-899e-ec18747dacef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:19.772119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2002cf-71f1-4965-899e-ec18747dacef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:19.772560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2002cf-71f1-4965-899e-ec18747dacef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:19.837621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4b2002cf-71f1-4965-899e-ec18747dacef None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:19.847903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b2002cf-71f1-4965-899e-ec18747dacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:19.966731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:20.015409 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1477/3002] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:20.091667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:20.093246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:20.157122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:20.229699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063fc83-444d-4123-922a-1c5df60c188e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:20.234125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:20.235972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:20.236304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:20.236582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:20.258737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:20.262410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:20.284759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1526/3003] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:21.450864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:21.451954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:21.452453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:21.483459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] MFA Rules not processed for user `3aa253ded2304ac3917a1788b6a1500b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:21.486334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:21.710799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:21.713114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:21.730200 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1478/3004] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:21.740801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4cd95-598d-4cbf-98ad-119eecc40f0b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:21.755342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:21.756276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:21.757682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:21.758626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] PATH_INFO: `/v3/projects/7aea38006c4548c48e56248dae811780` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:21.772557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aea38006c4548c48e56248dae811780)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:21.807475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:21.992323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aea38006c4548c48e56248dae811780', 'request_id': 'req-dc1910cf-f159-4150-b1de-ced02753cc7a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.012332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aea38006c4548c48e56248dae811780', 'request_id': 'req-dc1910cf-f159-4150-b1de-ced02753cc7a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.027428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aea38006c4548c48e56248dae811780', 'request_id': 'req-dc1910cf-f159-4150-b1de-ced02753cc7a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.051022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] The token cache is being invalidate because project 7aea38006c4548c48e56248dae811780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:22.052188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.053956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.058467 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1527/3005] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:21 2022] DELETE /identity/v3/projects/7aea38006c4548c48e56248dae811780 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:22.062631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:22.085679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26687bcd-39d5-46e8-b6ff-efc70a856258 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:22.240249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:22.241743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:22.242030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:22.242302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] PATH_INFO: `/v3/users/152d1f77bdbb42078b8018ae96d44b61` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:22.244668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=152d1f77bdbb42078b8018ae96d44b61)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:22.248784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:22.360552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152d1f77bdbb42078b8018ae96d44b61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.362672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152d1f77bdbb42078b8018ae96d44b61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.379261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152d1f77bdbb42078b8018ae96d44b61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.407322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152d1f77bdbb42078b8018ae96d44b61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.410712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6124ef9-74fa-43bc-b304-2691673a1f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152d1f77bdbb42078b8018ae96d44b61'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:22.463465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1479/3006] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:22 2022] DELETE /identity/v3/users/152d1f77bdbb42078b8018ae96d44b61 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:23.248091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:23.250203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:23.273354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1910cf-f159-4150-b1de-ced02753cc7a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:23.422156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:23.422873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:23.423337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:23.423620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] PATH_INFO: `/v3/projects/c48e668375fe4fc89ff46adafece5fd9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:23.427150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48e668375fe4fc89ff46adafece5fd9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:23.429177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:23.570626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48e668375fe4fc89ff46adafece5fd9', 'request_id': 'req-5a2d8e81-13f0-47bb-be96-eedbc52483a8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:23.574459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:23.611079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48e668375fe4fc89ff46adafece5fd9', 'request_id': 'req-5a2d8e81-13f0-47bb-be96-eedbc52483a8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:23.627130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48e668375fe4fc89ff46adafece5fd9', 'request_id': 'req-5a2d8e81-13f0-47bb-be96-eedbc52483a8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:23.673218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] The token cache is being invalidate because project c48e668375fe4fc89ff46adafece5fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:23.673537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:23.681273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a2d8e81-13f0-47bb-be96-eedbc52483a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:23.683674 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1528/3007] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:23 2022] DELETE /identity/v3/projects/c48e668375fe4fc89ff46adafece5fd9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:23.688314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:23.688849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:23.688849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:23.727803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:23.744913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:23.746261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:23.929283 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1480/3008] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:23.930053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:23.930336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:23.930709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.011385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:24.013464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:24.034026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1529/3009] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:24.038485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.055470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b758337-5cbc-442a-8125-3a1d866a1584 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.070378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.071364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.071892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.072441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.073558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:24.074089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.076006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.102147 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1481/3010] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:56:24.107132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.125994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fd36d-5183-47d8-a815-63523944ad82 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.137381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.139104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.140115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.140561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.148185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.152581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.256441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1530/3011] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:24 2022] POST /identity/v3/projects => generated 403 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:24.259981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.286931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28290a85-31b8-4503-8d6f-9ae982f94d36 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.494901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.494901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.495292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.495404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.496053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.498002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.567640 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1482/3012] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:24 2022] POST /identity/v3/users => generated 302 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:24.569601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.590178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4eab1a-2539-48ba-9852-4493aa61f20f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.615849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.619492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.620019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.620157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.620922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.633910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.656679 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1531/3013] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:24.662301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.686004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffa38-0e28-49c1-99ea-d8fa46811846 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.699168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.699868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.700055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.713677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.717964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.726693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bb8ae-f560-4157-b6cc-9ecf1ef691d5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.767364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:24.778168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1483/3014] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:24.817650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595c54-43ea-4d5e-a1a3-a04691aa4ea9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:24.829478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:24.830161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.830443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.830855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] PATH_INFO: `/v3/projects/a6d88a97354b45e7a010e925822ea890/users/304d9738234046a19b17b4b87f8ffa00/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.851257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d88a97354b45e7a010e925822ea890, user_id=304d9738234046a19b17b4b87f8ffa00, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:24.854488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:24.873293 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1532/3015] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:24 2022] PUT /identity/v3/projects/a6d88a97354b45e7a010e925822ea890/users/304d9738234046a19b17b4b87f8ffa00/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:24.884615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:24.885142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:24.885602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:24.938919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] MFA Rules not processed for user `304d9738234046a19b17b4b87f8ffa00`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:24.941974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:25.119003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1484/3016] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:25.122131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:25.265512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4c91d-a65f-446e-84eb-a98b36f6e2bf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:25.272659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:25.273574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:25.274005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:25.274449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:25.275440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:25.277798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:25.333029 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1533/3017] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:25 2022] POST /identity/v3/projects => generated 399 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:25.336789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:25.342676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:25.357085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc9952-a9ec-4c26-9798-b24a3aea2689 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:25.463283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:25.464093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:25.465401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:25.465401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:25.467253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:25.474385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:25.517502 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1485/3018] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:25 2022] POST /identity/v3/users => generated 306 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:25.521753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:25.541816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1dcd-25ca-42f8-95c7-a5e3416a9e59 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:25.556463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:25.557015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:25.557293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:25.557561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:25.558202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:25.559852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:25.580211 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1534/3019] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:25.585165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:25.603525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c85b41-08e4-4c0f-bee4-a4714bfc9738 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:25.612913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:25.613905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:25.614433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:25.614973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] PATH_INFO: `/v3/projects/550e98c5707b43b09841a0d85d9412c3/users/95006050314a43eeb9a2d08da2384cb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:25.664322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550e98c5707b43b09841a0d85d9412c3, user_id=95006050314a43eeb9a2d08da2384cb1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:25.667531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:25.726778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1486/3020] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:25 2022] PUT /identity/v3/projects/550e98c5707b43b09841a0d85d9412c3/users/95006050314a43eeb9a2d08da2384cb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:25.730995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:25.749364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fa274-3e1e-49bd-8370-c2835d1b6edd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:25.916136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:25.959857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:25.961214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:25.961988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:25.962393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:25.963676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:25.971295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728ef63-55b1-4cc7-9a1f-85d0ebcc60da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:26.009385 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1535/3021] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:26.014401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:26.041924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153afd03-50f5-4e78-9495-a2b2edbf7f38 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:26.064377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:26.064664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:26.065107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:26.065439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] PATH_INFO: `/v3/projects/550e98c5707b43b09841a0d85d9412c3/users/95006050314a43eeb9a2d08da2384cb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:26.069217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=550e98c5707b43b09841a0d85d9412c3, user_id=95006050314a43eeb9a2d08da2384cb1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:26.079196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aa6c2-f6ce-437e-82f8-dc5a0d830870 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:26.112623 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1487/3022] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:26 2022] PUT /identity/v3/projects/550e98c5707b43b09841a0d85d9412c3/users/95006050314a43eeb9a2d08da2384cb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:26.124166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:26.124314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:26.124653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:26.185904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] MFA Rules not processed for user `95006050314a43eeb9a2d08da2384cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:26.187367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:26.355720 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1536/3023] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:26 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:27.354303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:27.355228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:27.355696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:27.400879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] MFA Rules not processed for user `76b517a69e1f4dd0895104befebc128d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:27.403487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:27.561917 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1488/3024] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:28.724700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:28.727024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:28.852045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092cfbe-d3f7-4b46-961e-b415b53e5ce5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:28.859463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:28.860925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:28.861579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:28.863511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:28.879389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:28.882159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:28.896412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1537/3025] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:28 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:28.968675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:28.990882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4704-6863-4776-aad6-948a1f81b1a8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:29.007047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:29.007866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:29.007994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:29.008237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:29.042431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:29.044426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:29.052652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:29.055188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:29.076406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1489/3026] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:29.187764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:29.198813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b3b50-31cf-495c-9c52-0a9764bffe8d service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:29.205215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:29.206429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:29.206972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:29.207448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] PATH_INFO: `/v3/users/6c381100cbcb4f128438e334e9beca6b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:29.208932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c381100cbcb4f128438e334e9beca6b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:29.211239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:29.292192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c381100cbcb4f128438e334e9beca6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:29.294160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c381100cbcb4f128438e334e9beca6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:29.306293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c381100cbcb4f128438e334e9beca6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:29.344327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c381100cbcb4f128438e334e9beca6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:29.345717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7796a4e8-4362-4391-b51c-243013d5809d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c381100cbcb4f128438e334e9beca6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:29.368793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1538/3027] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:29 2022] DELETE /identity/v3/users/6c381100cbcb4f128438e334e9beca6b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:30.544346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:30.570123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed244da-d031-484b-8b50-ce2324350770 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:30.604734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:30.668690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:30.669628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:30.670158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:30.670324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] PATH_INFO: `/v3/projects/9b6e724ba5a849a2ac33fba2b3868ccc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:30.672417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6e724ba5a849a2ac33fba2b3868ccc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:30.677660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:30.764053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6e724ba5a849a2ac33fba2b3868ccc', 'request_id': 'req-e544fe83-5f2e-4d98-b0ac-64e292086551'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:30.775037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6e724ba5a849a2ac33fba2b3868ccc', 'request_id': 'req-e544fe83-5f2e-4d98-b0ac-64e292086551'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:30.785013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6e724ba5a849a2ac33fba2b3868ccc', 'request_id': 'req-e544fe83-5f2e-4d98-b0ac-64e292086551'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:30.829392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] The token cache is being invalidate because project 9b6e724ba5a849a2ac33fba2b3868ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:30.832177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:30.833926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e544fe83-5f2e-4d98-b0ac-64e292086551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:30.837575 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1490/3028] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:30 2022] DELETE /identity/v3/projects/9b6e724ba5a849a2ac33fba2b3868ccc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:30.842545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:30.843056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:30.843530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:30.920410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:30.923943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:31.043665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1539/3029] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:31.054573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.055043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.055500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.096617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:31.099101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:31.119098 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1491/3030] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:31.119628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.133951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65379359-5137-4848-a393-19c66fa4dfe2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.145030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.146707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.149038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.151004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.153841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:31.158697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.162931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:31.208230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.215706 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1540/3031] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:56:31.249897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc92e28-094a-409c-8b8a-1842ab283aed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.275038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.275940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.301728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.303562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.305444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.310031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:31.392576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.401526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1492/3032] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:31 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:31.411785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae41bc-2e5a-446b-8a03-d85e3050001a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.663290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.664345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.664793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.665229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.666034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.669389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:31.705327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:31.737811 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1541/3033] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:31 2022] POST /identity/v3/users => generated 291 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:31.741884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.760826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfd70a-43b9-497d-be07-8efffc178b8f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.766799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.767473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.767929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.768972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.770332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.772025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:31.810712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1493/3034] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:31.813679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.834955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac079f-dd87-420f-9471-5813932e3b28 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.844888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.846863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.847377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.847830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.848643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.850436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ab120-b5ea-434c-b7a9-b00cce92cd98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:31.871519 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1542/3035] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:31.873348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:31.898330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1fa4c-0a50-490a-9563-53748826ed24 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:31.927357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:31.928069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:31.928541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:31.930070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] PATH_INFO: `/v3/projects/7c8ce066fefe47a69e2bf6d625c01eb7/users/5c77dbc1dbd748668e76f2bebc0220d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:31.971750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8ce066fefe47a69e2bf6d625c01eb7, user_id=5c77dbc1dbd748668e76f2bebc0220d1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:31.976553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:32.013441 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1494/3036] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:31 2022] PUT /identity/v3/projects/7c8ce066fefe47a69e2bf6d625c01eb7/users/5c77dbc1dbd748668e76f2bebc0220d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:32.026405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798fe2-1579-4786-adad-c1a7dc69785b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:32.026737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798fe2-1579-4786-adad-c1a7dc69785b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:32.027609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798fe2-1579-4786-adad-c1a7dc69785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:32.097421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-65798fe2-1579-4786-adad-c1a7dc69785b None None] MFA Rules not processed for user `5c77dbc1dbd748668e76f2bebc0220d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:32.099860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65798fe2-1579-4786-adad-c1a7dc69785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:32.298546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1543/3037] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:32.321526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:32.326643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:32.463386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e23de-bcb9-4178-8a14-02bd74f74c60 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:32.474008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:32.474688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:32.474968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:32.475241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:32.495309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:32.496795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:32.519310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:32.530189 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1495/3038] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:32.848592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2dea6-6182-4e8f-9f2a-8133d6dd0137 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:32.849581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2dea6-6182-4e8f-9f2a-8133d6dd0137 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:32.850024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2dea6-6182-4e8f-9f2a-8133d6dd0137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:32.895234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-30f2dea6-6182-4e8f-9f2a-8133d6dd0137 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:32.896814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-30f2dea6-6182-4e8f-9f2a-8133d6dd0137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:33.035608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1544/3039] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:33.134651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:33.136877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:33.318306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132832-33e4-430a-b610-f79a715fed3c service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:33.336897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:33.337842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:33.338281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:33.338904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:33.359985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:33.362885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:33.377246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1496/3040] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:56:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:56:35.551578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:35.552120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:35.552120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:35.600311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] MFA Rules not processed for user `76b517a69e1f4dd0895104befebc128d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:35.603417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:35.751947 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1545/3041] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:37.153375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:37.156280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:37.400808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7af-5dd6-473e-89b4-435c6eb99cca service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:37.412950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:37.414783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:37.414994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:37.415876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] PATH_INFO: `/v3/users/e6b8fe778b5447e4be355f476052ba5c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:37.417648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b8fe778b5447e4be355f476052ba5c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:37.420851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:37.504848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b8fe778b5447e4be355f476052ba5c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:37.505763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b8fe778b5447e4be355f476052ba5c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:37.524708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b8fe778b5447e4be355f476052ba5c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:37.542817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b8fe778b5447e4be355f476052ba5c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:37.543986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7131129-ac8f-4eb6-af59-3d07f9bb00bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b8fe778b5447e4be355f476052ba5c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:37.575658 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1497/3042] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:37 2022] DELETE /identity/v3/users/e6b8fe778b5447e4be355f476052ba5c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:38.387204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:38.425030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e09952-7164-4563-aaf4-7dcd7bee6232 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:38.532310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:38.668454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:38.669337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:38.669824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:38.670296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] PATH_INFO: `/v3/projects/bb818297b8a64e7ababc5eff3f4a1784` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:38.671811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb818297b8a64e7ababc5eff3f4a1784)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:38.677611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:38.775731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb818297b8a64e7ababc5eff3f4a1784', 'request_id': 'req-29d92fda-93e7-4841-a19f-ca2f5733230d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:38.805530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb818297b8a64e7ababc5eff3f4a1784', 'request_id': 'req-29d92fda-93e7-4841-a19f-ca2f5733230d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:38.835951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb818297b8a64e7ababc5eff3f4a1784', 'request_id': 'req-29d92fda-93e7-4841-a19f-ca2f5733230d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:38.866814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] The token cache is being invalidate because project bb818297b8a64e7ababc5eff3f4a1784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:38.867523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:38.868481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29d92fda-93e7-4841-a19f-ca2f5733230d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:38.872155 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1546/3043] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:38 2022] DELETE /identity/v3/projects/bb818297b8a64e7ababc5eff3f4a1784 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:38.881131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:38.881973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:38.882126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:38.916646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:38.918412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:38.926428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:39.034010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1498/3044] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:39.042061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.042354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.042668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.131256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:39.137133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.173106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1547/3045] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:39.178376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.189536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5154f59-4848-49ad-92f0-1fefb866df7a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.196750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.197706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.198190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.198685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.200295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:39.200766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.203369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.241109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1499/3046] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:56:39.245711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.259437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862e20f-fd28-41af-937c-7f3c56ddde04 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.270985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.272406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.273925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.274756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.278328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.280608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.318569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1548/3047] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:39 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:39.321456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.333315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61891bf-839c-4b64-a767-2bee96313641 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.414526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.415587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.415587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.415832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.416536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.418550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.450387 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1500/3048] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:39 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:39.455011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.470301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bb11a-e539-4bad-a314-c6b6c063d843 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.477227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.477838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.478806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.478933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.480259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.482221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.498160 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1549/3049] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:39.499639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.522598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d13d2-bf7f-49e0-ad7c-abe013ec7693 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.531788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.532425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.532701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.532970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.534188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.536465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.537264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.540189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.562834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1501/3050] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:39.563828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.577389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb2fc0-450c-4613-a04b-7a35e97b8016 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.584355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.585573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.585573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.585821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] PATH_INFO: `/v3/projects/45916bd2a2e847b5ae05290dd106b04c/users/56064ff9075243879d7bce37e3e547e7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.605476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45916bd2a2e847b5ae05290dd106b04c, user_id=56064ff9075243879d7bce37e3e547e7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.609315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1128f64-e7fb-4d74-9c37-8f12f3a40b12 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.628077 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1502/3051] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:39 2022] PUT /identity/v3/projects/45916bd2a2e847b5ae05290dd106b04c/users/56064ff9075243879d7bce37e3e547e7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:39.633937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.634232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.634773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.666616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:39.668415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.688844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f94a3-5617-4c10-b0c4-f38a94d1e65d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.696245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:39.742588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:39.764005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1503/3052] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:56:39.772256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.772980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.773339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.773722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.774736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.775761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.776120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.850013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.850289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59349271-ba89-4eb2-8d67-50fa2e84f2b0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.851769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.853884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.854475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.854783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.855051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.862792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1550/3053] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:39 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:56:39.866480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.867951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50712451-98c3-4000-b01d-73f08dca8d32 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:39.884615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:39.887213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:39.897967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1504/3054] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:56:39.920882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897abd-cedd-42ea-abbd-a927b305bec0 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:39.927947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:39.928836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:39.928972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:39.929299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] PATH_INFO: `/v3/users/304d9738234046a19b17b4b87f8ffa00` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:39.930632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=304d9738234046a19b17b4b87f8ffa00)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:39.932874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:40.019450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304d9738234046a19b17b4b87f8ffa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:40.020457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304d9738234046a19b17b4b87f8ffa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:40.032620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304d9738234046a19b17b4b87f8ffa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:40.052488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304d9738234046a19b17b4b87f8ffa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:40.053555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304d9738234046a19b17b4b87f8ffa00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:40.078193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1551/3055] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:39 2022] DELETE /identity/v3/users/304d9738234046a19b17b4b87f8ffa00 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:40.092713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:40.094115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:40.095099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:40.142275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:40.145062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:40.287361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1505/3056] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:40.348344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:40.349969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:40.402127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:40.532173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e41f2-079c-4687-911f-ea12936a50cb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:40.568578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:40.571918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:40.572539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:40.573010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:40.600301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:40.602549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:40.621581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1552/3057] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:41.767673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:41.770195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:41.805258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46427e9-85fd-4b2e-af0d-269e92c288be None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:41.817920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:41.819015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:41.821158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:41.827052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] PATH_INFO: `/v3/projects/a6d88a97354b45e7a010e925822ea890` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:41.828521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6d88a97354b45e7a010e925822ea890)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:41.830419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:41.948850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6d88a97354b45e7a010e925822ea890', 'request_id': 'req-d82f2219-1e29-4a8c-b60a-6c492719c53a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:41.960371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6d88a97354b45e7a010e925822ea890', 'request_id': 'req-d82f2219-1e29-4a8c-b60a-6c492719c53a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:41.972702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6d88a97354b45e7a010e925822ea890', 'request_id': 'req-d82f2219-1e29-4a8c-b60a-6c492719c53a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.000838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] The token cache is being invalidate because project a6d88a97354b45e7a010e925822ea890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:42.001119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.004883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.010076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1506/3058] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:41 2022] DELETE /identity/v3/projects/a6d88a97354b45e7a010e925822ea890 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:42.014318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:42.037097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c40648-7b92-4f53-bce0-94ea5391664d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:42.194057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:42.204538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:42.204976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:42.205346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] PATH_INFO: `/v3/users/95006050314a43eeb9a2d08da2384cb1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:42.207546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95006050314a43eeb9a2d08da2384cb1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:42.210155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:42.322168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95006050314a43eeb9a2d08da2384cb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.325205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95006050314a43eeb9a2d08da2384cb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.337231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95006050314a43eeb9a2d08da2384cb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.369334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95006050314a43eeb9a2d08da2384cb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.382044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1c2adc96-7f53-4711-a4de-51020dcd7ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95006050314a43eeb9a2d08da2384cb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:42.417860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1553/3059] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:42 2022] DELETE /identity/v3/users/95006050314a43eeb9a2d08da2384cb1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:43.700035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:43.702152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:43.727811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f2219-1e29-4a8c-b60a-6c492719c53a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:43.762201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:43.856612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:43.857482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:43.857830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:43.858194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] PATH_INFO: `/v3/projects/550e98c5707b43b09841a0d85d9412c3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:43.859768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550e98c5707b43b09841a0d85d9412c3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:43.863217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:43.978038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550e98c5707b43b09841a0d85d9412c3', 'request_id': 'req-be1bef51-ef07-4aab-953e-180032c45426'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:43.986081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550e98c5707b43b09841a0d85d9412c3', 'request_id': 'req-be1bef51-ef07-4aab-953e-180032c45426'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:44.003712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550e98c5707b43b09841a0d85d9412c3', 'request_id': 'req-be1bef51-ef07-4aab-953e-180032c45426'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:44.028721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] The token cache is being invalidate because project 550e98c5707b43b09841a0d85d9412c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:44.029194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:44.030149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-be1bef51-ef07-4aab-953e-180032c45426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:44.035905 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1507/3060] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:43 2022] DELETE /identity/v3/projects/550e98c5707b43b09841a0d85d9412c3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:44.037968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.038459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.038777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.076496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:44.077760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:44.148036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:44.204415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1554/3061] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:44.212035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.212613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.212994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.256219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:44.257912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:44.280216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1508/3062] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:44.285011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.296484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302dd3c-68be-40b1-8c40-94423f8094a8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.302757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.303722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.304463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.304959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.305834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:44.306377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.309253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.329724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.335344 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1555/3063] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:56:44.342860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9d1a3-fb87-49da-ae4f-3fb1330147c2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.368660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.369562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.370739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.372803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.373578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.386663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.436787 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1509/3064] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:44 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:44.441889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.458782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d298d-0e97-45ce-8522-cd6ff0adaa61 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.560079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.563288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.563610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.563893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.564550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.568710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.607232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1556/3065] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:44 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:44.611971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.623260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98f949-6ff0-4a6c-823a-14fa457df931 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.649094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.650004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.650598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.653107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.654556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.656415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.672865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1510/3066] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:56:44.676465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.689243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23adf2a-93e9-4936-bb35-db1fc9426c4a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.725014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.725139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.725366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.725614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.726336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.734225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423c9d3-f08e-4d25-82ae-50dbf272b116 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.766964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1557/3067] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:44.768230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:44.787601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bae304-7c5f-491c-80fa-9af954640935 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:44.801544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:44.802870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.803179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.803568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] PATH_INFO: `/v3/projects/749824e2c16946a58801200784fa0480/users/9dd09a0889c44ce18f0c361c7c661465/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:44.823199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] RBAC: Authorizing `identity:create_grant(project_id=749824e2c16946a58801200784fa0480, user_id=9dd09a0889c44ce18f0c361c7c661465, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:44.826178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:44.861354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1511/3068] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:44 2022] PUT /identity/v3/projects/749824e2c16946a58801200784fa0480/users/9dd09a0889c44ce18f0c361c7c661465/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:44.940599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:44.942830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:44.943574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:45.021703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] MFA Rules not processed for user `9dd09a0889c44ce18f0c361c7c661465`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:45.023752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:45.220736 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1558/3069] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:45.225792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:45.331482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:45.358689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd952e-4fbe-4ad7-b9f5-32f56987820a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:45.365212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:45.367485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:45.368214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:45.368877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:45.370479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:45.373171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:45.428617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1512/3070] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:45.433384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:45.449232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6e1c-e102-4841-9411-7a6eac8a83f8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:45.494322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:45.542387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:45.544562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:45.545651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:45.545944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:45.547234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:45.555506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:45.591149 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1559/3071] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:45 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:45.595253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:45.610943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5c835-6f87-44bb-9a6a-9d65dc1ec15e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:45.623868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:45.624255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:45.624538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:45.624811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:45.625656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:45.633118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:45.654276 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1513/3072] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:45.659000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:45.677533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc23469-56de-4ac2-a7d6-666c48555da7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:45.697750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:45.698587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:45.698960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:45.699720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] PATH_INFO: `/v3/projects/419492810b3341d19f1f05c4ebed60cf/users/2f607c3c0fcf4e97b7208cb7034b0ec4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:45.763284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419492810b3341d19f1f05c4ebed60cf, user_id=2f607c3c0fcf4e97b7208cb7034b0ec4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:45.770881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:45.807099 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1560/3073] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:45 2022] PUT /identity/v3/projects/419492810b3341d19f1f05c4ebed60cf/users/2f607c3c0fcf4e97b7208cb7034b0ec4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:45.811412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:45.844043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c18902-0690-485f-932c-2e85b7155fed admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:46.026847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:46.028258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:46.028534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:46.028806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:46.029458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:46.031124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd57b4-9d7c-422a-bfb6-9ab323accb22 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:46.071560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1514/3074] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:46.075769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:46.093733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461518a-84d4-4be9-ab6e-cf14152b9341 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:46.107009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:46.107686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:46.107964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:46.108233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] PATH_INFO: `/v3/projects/419492810b3341d19f1f05c4ebed60cf/users/2f607c3c0fcf4e97b7208cb7034b0ec4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:46.120828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=419492810b3341d19f1f05c4ebed60cf, user_id=2f607c3c0fcf4e97b7208cb7034b0ec4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:46.124375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdbea2-c098-44b5-a762-6bbfe35b5ebb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:46.155306 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1561/3075] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:46 2022] PUT /identity/v3/projects/419492810b3341d19f1f05c4ebed60cf/users/2f607c3c0fcf4e97b7208cb7034b0ec4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:46.166428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:46.166913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:46.167119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:46.224806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] MFA Rules not processed for user `2f607c3c0fcf4e97b7208cb7034b0ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:46.228245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:46.409953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1515/3076] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:46.421372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b2ce-d3ec-4536-b0dd-bb1b64ee5415 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:46.421922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b2ce-d3ec-4536-b0dd-bb1b64ee5415 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:46.421922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b2ce-d3ec-4536-b0dd-bb1b64ee5415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:46.461843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dab5b2ce-d3ec-4536-b0dd-bb1b64ee5415 None None] MFA Rules not processed for user `9dd09a0889c44ce18f0c361c7c661465`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:46.463230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dab5b2ce-d3ec-4536-b0dd-bb1b64ee5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:46.621929 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1562/3077] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:47.147841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:47.154957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:47.239440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:47.341219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ef1c-74d4-450e-b0e0-3f82b74e6b58 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:47.379387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:47.380102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:47.380587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:47.381107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:47.404225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:47.406098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:47.430150 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1516/3078] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:48.660197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:48.661255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:48.661747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:48.699951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] MFA Rules not processed for user `56064ff9075243879d7bce37e3e547e7`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:48.701478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:48.811791 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1563/3079] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:48.812630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:48.814770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:48.926545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9417f5-2319-4c6c-aced-543e62dea802 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:48.937640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:48.938538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:48.938943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:48.939288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:48.940687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:48.943811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:48.982555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:48.992701 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1517/3080] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:48 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:56:48.997259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce4dcb-a1ba-4365-939f-9efbd90ab2e6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:49.024840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:49.076617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:49.077552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:49.077552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:49.077778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:49.087402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:49.089420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:49.130318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1564/3081] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:48 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:49.134851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:49.148772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f00215-647e-42cd-b116-ceb55dcf7e50 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:49.154218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:49.154902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:49.155182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:49.155457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:49.156753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:49.158471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:49.171555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1518/3082] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:49.176616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:49.189927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ba5d-7de3-47c7-a560-455daba170f1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:49.197430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:49.198122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:49.198405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:49.198865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] PATH_INFO: `/v3/projects/86a66901b4074ab99c4f7acb86d25e08/users/8c719e43f261418593e57d05bcff7701/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:49.234262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a66901b4074ab99c4f7acb86d25e08, user_id=8c719e43f261418593e57d05bcff7701, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:49.240215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:49.265425 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1565/3083] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:49 2022] PUT /identity/v3/projects/86a66901b4074ab99c4f7acb86d25e08/users/8c719e43f261418593e57d05bcff7701/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:49.268192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:49.281058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a728c-beb9-4212-b641-a088127ce556 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:49.445466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:49.446488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:49.447538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:49.447837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:49.448504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:49.450252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8cd16-36ed-4c37-8609-ae365d06b383 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:49.483983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1519/3084] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:49.487713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:49.503184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f4e-2ae2-4689-8ea3-79e1ad4e7ec9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:49.524421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:49.525102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:49.525381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:49.525648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] PATH_INFO: `/v3/projects/86a66901b4074ab99c4f7acb86d25e08/users/8c719e43f261418593e57d05bcff7701/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:49.527876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a66901b4074ab99c4f7acb86d25e08, user_id=8c719e43f261418593e57d05bcff7701, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:49.530662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:49.558901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1566/3085] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:49 2022] PUT /identity/v3/projects/86a66901b4074ab99c4f7acb86d25e08/users/8c719e43f261418593e57d05bcff7701/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:56:50.454278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:50.455178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:50.455674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:50.513073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:50.514087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:50.656727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:50.752216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1520/3086] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:50.769136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:50.771264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:50.929309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463173-8374-448c-b5a4-38f25d4b13fd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:50.939522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:50.942268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:50.942560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:50.942840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:50.969507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:50.976066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384368f-f906-4d33-882d-fadc1fb6e3b0 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:50.990901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1567/3087] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:56:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:56:51.178894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:51.181121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:51.349229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8228140-43f7-4701-941a-26a2a96a26a4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:51.356786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:51.357432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:51.357707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:51.357975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] PATH_INFO: `/v3/users/5c77dbc1dbd748668e76f2bebc0220d1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:51.359189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c77dbc1dbd748668e76f2bebc0220d1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:51.361392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:51.430623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c77dbc1dbd748668e76f2bebc0220d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:51.431354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c77dbc1dbd748668e76f2bebc0220d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:51.443542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c77dbc1dbd748668e76f2bebc0220d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:51.452758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c77dbc1dbd748668e76f2bebc0220d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:51.453619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c77dbc1dbd748668e76f2bebc0220d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:51.500452 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1521/3088] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:51 2022] DELETE /identity/v3/users/5c77dbc1dbd748668e76f2bebc0220d1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:51.506012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:51.506300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:51.506595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:51.535612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:51.537467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:51.701386 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1568/3089] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:51.727030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:51.729783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:51.819522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3985f-6a82-4e7a-bf8b-b5a6d179e703 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:51.825238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:51.825972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:51.826343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:51.826785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:51.840939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:51.843083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a40231-bd1d-4ed9-a67a-482af7fe968d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:51.858193 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1522/3090] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:53.243924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:53.245599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:53.258382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:53.270632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fdae0-0a59-40f1-9442-2f72f286816e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:53.288257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:53.288919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:53.289201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:53.289480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] PATH_INFO: `/v3/projects/7c8ce066fefe47a69e2bf6d625c01eb7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:53.290671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c8ce066fefe47a69e2bf6d625c01eb7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:53.292847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:53.388034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c8ce066fefe47a69e2bf6d625c01eb7', 'request_id': 'req-7fe434ca-f71e-41f7-bd51-17d9c6a47417'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:53.427168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c8ce066fefe47a69e2bf6d625c01eb7', 'request_id': 'req-7fe434ca-f71e-41f7-bd51-17d9c6a47417'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:53.453221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c8ce066fefe47a69e2bf6d625c01eb7', 'request_id': 'req-7fe434ca-f71e-41f7-bd51-17d9c6a47417'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:53.481450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] The token cache is being invalidate because project 7c8ce066fefe47a69e2bf6d625c01eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:53.482179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:53.483519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7fe434ca-f71e-41f7-bd51-17d9c6a47417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:53.506057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1569/3091] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:53 2022] DELETE /identity/v3/projects/7c8ce066fefe47a69e2bf6d625c01eb7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:53.511023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:53.511486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:53.511486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:53.522608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:53.557298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:53.558593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:53.671736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1523/3092] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:53.679789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:53.680335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:53.680563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:53.729921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:53.732483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:53.771581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1570/3093] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:53.773349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:53.804554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c3da2-c779-490a-a7ba-e3a8f1e655a7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:53.819949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:53.820705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:53.821834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:53.822221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:53.824796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:53.824796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:53.827763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:53.890625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1524/3094] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:56:53.895826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:53.920809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc85125-75d7-4e14-961b-4cb9a978bae1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:53.938250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:53.962824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:53.963880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:53.964168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:53.966854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:53.972612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:54.053139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1571/3095] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:53 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:54.056274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:54.074729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428c859-7896-4397-aafb-8f26dc17061b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:54.220197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:54.221374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:54.221644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:54.221766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:54.222405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:54.224587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:54.262030 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1525/3096] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:54 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:56:54.265895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:54.281876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad7a68-11e3-45bb-a8b6-d84de3219ea2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:54.324626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:54.327031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:54.327376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:54.328050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:54.328930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:54.333555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:54.377790 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1572/3097] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:54.378192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:54.406597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083375d3-8e25-4809-a7fd-75412a89c632 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:54.424242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:54.425812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:54.426460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:54.426864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:54.427665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:54.429356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0d905-10e9-4a6a-bbbd-e2d0d415a903 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:54.479569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1526/3098] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:56:54.481118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:54.498472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3339af6-4a37-492c-bfcd-488512f0ce08 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:54.504383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:54.505833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:54.506334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:54.506862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] PATH_INFO: `/v3/projects/13b6517666a849e2811e98f9b0cd764a/users/cbd3b7d6a7744bfd9a9b2d3f804a9f0b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:54.527291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b6517666a849e2811e98f9b0cd764a, user_id=cbd3b7d6a7744bfd9a9b2d3f804a9f0b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:54.530927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:54.589900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1573/3099] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:54 2022] PUT /identity/v3/projects/13b6517666a849e2811e98f9b0cd764a/users/cbd3b7d6a7744bfd9a9b2d3f804a9f0b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:54.601525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:54.602710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:54.603539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:54.669033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] MFA Rules not processed for user `cbd3b7d6a7744bfd9a9b2d3f804a9f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:54.671152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:54.838623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:54.853170 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1527/3100] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:54.961265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:54.964395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:55.178198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dac62-66c2-41c8-974d-930d7e1987aa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:55.192609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:55.194709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:55.195107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:55.195507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:55.256215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:55.258103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31beb0-977f-4fe2-8aeb-a69f58e45cae service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:55.315961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1574/3101] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:55.359551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:55.520710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d1e9e-8a88-4400-b8f8-0b3ba8ffae1e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:55.528905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:55.529634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:55.529979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:55.530320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] PATH_INFO: `/v3/users/cbd3b7d6a7744bfd9a9b2d3f804a9f0b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:55.532114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd3b7d6a7744bfd9a9b2d3f804a9f0b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:55.534970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:55.609421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd3b7d6a7744bfd9a9b2d3f804a9f0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:55.610754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd3b7d6a7744bfd9a9b2d3f804a9f0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:55.653383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd3b7d6a7744bfd9a9b2d3f804a9f0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:55.666114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd3b7d6a7744bfd9a9b2d3f804a9f0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:55.668415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd3b7d6a7744bfd9a9b2d3f804a9f0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:55.698443 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1528/3102] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:55 2022] DELETE /identity/v3/users/cbd3b7d6a7744bfd9a9b2d3f804a9f0b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:55.702872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:55.703159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:55.703437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:55.739851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:55.741205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:55.771580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:55.895302 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1575/3103] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:55.961108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:55.963079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:56.094408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b772-be21-487c-9658-b3776be5d1b9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:56.098656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:56.099280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:56.099557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:56.099996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:56.131466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:56.134108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63251550-8f2f-46ab-b46d-49fce754c425 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:56.166569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1529/3104] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:56:56.979714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:56.984807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:57.010082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b4fb-eb42-4ff1-a116-0a1b38667ad1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.020786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.022908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.025580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.026536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] PATH_INFO: `/v3/projects/13b6517666a849e2811e98f9b0cd764a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.029321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b6517666a849e2811e98f9b0cd764a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.033197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.118249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b6517666a849e2811e98f9b0cd764a', 'request_id': 'req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:57.130702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b6517666a849e2811e98f9b0cd764a', 'request_id': 'req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:57.141112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b6517666a849e2811e98f9b0cd764a', 'request_id': 'req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:57.159072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] The token cache is being invalidate because project 13b6517666a849e2811e98f9b0cd764a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:56:57.160053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:57.161179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d81b44d8-2ee5-43b9-ab07-07d57ac86b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:57.164651 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1576/3105] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:56:56 2022] DELETE /identity/v3/projects/13b6517666a849e2811e98f9b0cd764a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:57.169722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.170223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.170324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.222703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:57.226147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:57.250693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:57.410710 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1530/3106] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:57.418619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.419109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.419488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.451004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:57.452376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:57.463560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1577/3107] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:57.467675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.479967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174ce62-79b2-4922-936f-d5d1fbc93d90 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.491149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.491835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.492217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.492593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.493614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:56:57.495104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.514625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.534811 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1531/3108] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:56:57.539098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.553733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d081ef-e58a-4182-8cd0-c71c61371e57 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.559209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.560094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.560541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.560966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.561773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.564679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.622634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.622880 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1578/3109] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:56:57 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:57.637105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59b283-bb9c-46a8-8487-daa210e831ac admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.744277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.745311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.745311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.745541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.746066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.749993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.811290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1532/3110] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:56:57 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:56:57.815576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.835844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469302-eacc-4a12-8f87-b3eb4abf1b85 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.865214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.866024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.866936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.866936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.870738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.876201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.905773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1579/3111] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:57.907438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.931611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52463016-0f7e-4d33-a862-8a2510506f8f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:57.943398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:57.944357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:57.944839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:57.945313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:57.946192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:57.948041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8a840-468e-41bd-aca7-533206139677 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:57.978541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1533/3112] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:56:57.984223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:57.999568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98673252-5f0c-411d-81f0-d3ff3f424303 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:58.014140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:58.014868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:58.015216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:58.015522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] PATH_INFO: `/v3/projects/3fb03e599c594df3a7480ab9074ee4a9/users/83ec68388d7d4ace91cc2d6cd7f7d741/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:58.056717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fb03e599c594df3a7480ab9074ee4a9, user_id=83ec68388d7d4ace91cc2d6cd7f7d741, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:58.059829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:58.096836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1580/3113] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:56:57 2022] PUT /identity/v3/projects/3fb03e599c594df3a7480ab9074ee4a9/users/83ec68388d7d4ace91cc2d6cd7f7d741/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:58.113377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:58.113931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:58.114217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:58.175042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] MFA Rules not processed for user `83ec68388d7d4ace91cc2d6cd7f7d741`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:58.176301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:58.318112 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1534/3114] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:58.318792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:58.320713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:58.370997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:58.448675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d78e2-2211-4ed9-9f73-736d842aaa97 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:58.458133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:58.459891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:58.460291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:58.460662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:58.481454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:58.485626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0322f-abd9-4886-8638-a95d30afa33a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:58.501785 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1581/3115] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:56:59.183948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:59.215743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:56:59.331057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9d097-e797-4c93-be1c-4e032dfe0baa None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:56:59.351913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:56:59.352841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:59.353309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:59.353745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] PATH_INFO: `/v3/users/83ec68388d7d4ace91cc2d6cd7f7d741` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:59.355694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ec68388d7d4ace91cc2d6cd7f7d741)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:56:59.359337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:56:59.490148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ec68388d7d4ace91cc2d6cd7f7d741'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:59.491265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ec68388d7d4ace91cc2d6cd7f7d741'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:59.505485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ec68388d7d4ace91cc2d6cd7f7d741'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:59.532919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ec68388d7d4ace91cc2d6cd7f7d741'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:59.534259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ec68388d7d4ace91cc2d6cd7f7d741'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:56:59.580336 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1535/3116] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:56:59 2022] DELETE /identity/v3/users/83ec68388d7d4ace91cc2d6cd7f7d741 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:56:59.585529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0ac5-5c44-4e44-9003-6a4195ff2a84 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:56:59.588583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0ac5-5c44-4e44-9003-6a4195ff2a84 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:56:59.589135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c0ac5-5c44-4e44-9003-6a4195ff2a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:56:59.647579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-877c0ac5-5c44-4e44-9003-6a4195ff2a84 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:56:59.652438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-877c0ac5-5c44-4e44-9003-6a4195ff2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:56:59.818569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1582/3117] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:56:59.859432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:56:59.860899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:00.032313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58dadb-b33e-4f34-b9bb-5ed3494dddb8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:00.040716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:00.041446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.041793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.042134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.066771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:00.068413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:00.105175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1536/3118] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:00.210390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575476-4656-452a-a309-1d9afb6a1a01 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.210947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575476-4656-452a-a309-1d9afb6a1a01 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.211495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575476-4656-452a-a309-1d9afb6a1a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.277142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0e575476-4656-452a-a309-1d9afb6a1a01 None None] MFA Rules not processed for user `8c719e43f261418593e57d05bcff7701`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:00.278794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e575476-4656-452a-a309-1d9afb6a1a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:00.427914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1583/3119] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:00.599253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:00.600799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:00.624068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38902741-301f-42b0-98b3-7e994a400bd2 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:00.642436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:00.645977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.650466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.650783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] PATH_INFO: `/v3/projects/3fb03e599c594df3a7480ab9074ee4a9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.652019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fb03e599c594df3a7480ab9074ee4a9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:00.654245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:00.697512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:00.712686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fb03e599c594df3a7480ab9074ee4a9', 'request_id': 'req-4b662ad6-33f7-4321-bedb-620252c42187'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:00.718889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fb03e599c594df3a7480ab9074ee4a9', 'request_id': 'req-4b662ad6-33f7-4321-bedb-620252c42187'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:00.728488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fb03e599c594df3a7480ab9074ee4a9', 'request_id': 'req-4b662ad6-33f7-4321-bedb-620252c42187'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:00.740637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] The token cache is being invalidate because project 3fb03e599c594df3a7480ab9074ee4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:00.740938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:00.741584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b662ad6-33f7-4321-bedb-620252c42187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:00.743100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1537/3120] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:00 2022] DELETE /identity/v3/projects/3fb03e599c594df3a7480ab9074ee4a9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:00.748116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.748629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.748739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.768925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:00.770414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:00.814639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:00.857773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1584/3121] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:00.866375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.866612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.866861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.889602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:00.890939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:00.903194 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1538/3122] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:00.904643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:00.915508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a10ed7-025c-4d0e-a36d-677547a398db None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:00.920157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:00.920758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.921031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.921297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.921931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:57:00.922224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:00.923873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:00.939680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1585/3123] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:57:00.942047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:00.951193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dd7b5-a555-4761-930c-1e4eb5c96850 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:00.957056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:00.957651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:00.957924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:00.958187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:00.958949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:00.960853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:00.991116 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1539/3124] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:00 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:57:00.995340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.006205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0372c-2348-4080-943b-7f1a1b84d9db admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.092388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.092388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.092388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.092388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.092957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.092957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:01.142542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1586/3125] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:00 2022] POST /identity/v3/users => generated 296 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:57:01.145472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.156480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e57-0131-4e43-abbe-bf1da0b978d3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.165560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.166386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.167116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.167116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.168127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.173657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:01.184948 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1540/3126] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:01.189144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.201282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978831d-7d0a-4a1a-b334-8c1cdacc240a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.208468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.209049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.209330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.209602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.210249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.212437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f7f0-3044-4480-bb35-22b51c464811 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:01.226624 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1587/3127] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:01.231012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.245374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affce1aa-e548-4a8b-a1a1-22eabaab4149 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.252068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.253433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.253791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.254098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] PATH_INFO: `/v3/projects/7588851b153c4c0dbc7e63cfa1778497/users/2f062663eef74d63aea291eaa5f213a4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.272716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7588851b153c4c0dbc7e63cfa1778497, user_id=2f062663eef74d63aea291eaa5f213a4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.276678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:01.303155 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1541/3128] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:01 2022] PUT /identity/v3/projects/7588851b153c4c0dbc7e63cfa1778497/users/2f062663eef74d63aea291eaa5f213a4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:01.308474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.308791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.309064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.333525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:01.334805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:01.403741 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1588/3129] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:01.415170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.417295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:01.520557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1dcbb-93a8-4eb8-a76b-37e6728e7aab admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.525183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.525705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.525979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.526246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.540245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.542337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:01.558578 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1542/3130] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:01.851181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:01.853188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:01.899631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad40b38-49ac-4be3-9f18-69db117d4329 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:01.911465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:01.912249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:01.912531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:01.912804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] PATH_INFO: `/v3/users/302ac6f497cf4d3c8c638f5b4b49f723` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:01.916179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=302ac6f497cf4d3c8c638f5b4b49f723)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:01.922536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:02.020102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:02.033230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302ac6f497cf4d3c8c638f5b4b49f723'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.034393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302ac6f497cf4d3c8c638f5b4b49f723'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.067359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302ac6f497cf4d3c8c638f5b4b49f723'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.086897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302ac6f497cf4d3c8c638f5b4b49f723'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.087640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-799cf810-c9e7-40aa-b2c0-87540aae6b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302ac6f497cf4d3c8c638f5b4b49f723'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.130722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1589/3131] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:01 2022] DELETE /identity/v3/users/302ac6f497cf4d3c8c638f5b4b49f723 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:02.796163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:02.807181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd2d73-75ea-4ab2-b5b2-f150a1b50783 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:02.894032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:02.894796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:02.895034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:02.895309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] PATH_INFO: `/v3/projects/8afe75ffdb814e0ba23546cf3866ceac` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:02.896687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afe75ffdb814e0ba23546cf3866ceac)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:02.899071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:02.901467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:02.965628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afe75ffdb814e0ba23546cf3866ceac', 'request_id': 'req-2e444a26-1e92-4130-a019-b7e4c44fa965'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.972494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afe75ffdb814e0ba23546cf3866ceac', 'request_id': 'req-2e444a26-1e92-4130-a019-b7e4c44fa965'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:02.987542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afe75ffdb814e0ba23546cf3866ceac', 'request_id': 'req-2e444a26-1e92-4130-a019-b7e4c44fa965'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:03.010096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] The token cache is being invalidate because project 8afe75ffdb814e0ba23546cf3866ceac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:03.010639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:03.011360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2e444a26-1e92-4130-a019-b7e4c44fa965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:03.016373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1543/3132] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:02 2022] DELETE /identity/v3/projects/8afe75ffdb814e0ba23546cf3866ceac => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:03.018303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.019447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.019447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.060276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:03.066465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:03.165373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1590/3133] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:03.167717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.168071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.168456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.193838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:03.195041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:03.203575 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1544/3134] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:03.207655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.217729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c3981-873c-4d02-9483-be3095f12ccc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.223956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.225441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.225441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.225441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.225763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:57:03.225970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.227585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.240581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1591/3135] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:57:03.245145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.255592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868aa42-bc0d-4120-8fef-1de5b6293ea6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.262132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.263893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.263893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.263893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.264840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.266905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.306431 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1545/3136] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:03 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:03.310716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.339659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ac06b-e491-472b-abdc-c0db9c4f673f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.426635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.427465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.427991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.428368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.429273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.432495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.464935 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1592/3137] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:03 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:03.471068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.487123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1189a3-2509-4dbf-9d32-ae45b029f1f5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.494282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.495505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.495872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.496745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.497609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.499812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.514895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1546/3138] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:03.520503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.534743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c30be0-3c60-4230-8888-0dd7d4b25c30 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.540901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.542531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.542817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.543156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.543736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.545978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae07a87-1aac-4870-8da2-630e6d7faba3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.565027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1593/3139] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:03.568347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.594165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a676e8-b4c5-4594-8bfb-491ed6b80b33 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.604577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.605362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.606189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.606189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] PATH_INFO: `/v3/projects/265d7d88bb6f45708203ea74df8f3214/users/00585dbe7f814eeebda43417ca1d0ffa/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.623664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265d7d88bb6f45708203ea74df8f3214, user_id=00585dbe7f814eeebda43417ca1d0ffa, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.626616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.644860 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1547/3140] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:03 2022] PUT /identity/v3/projects/265d7d88bb6f45708203ea74df8f3214/users/00585dbe7f814eeebda43417ca1d0ffa/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:03.655500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.655782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.656055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.694474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] MFA Rules not processed for user `00585dbe7f814eeebda43417ca1d0ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:03.697536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:03.741237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:03.820912 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1594/3141] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:03.837360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:03.839835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:03.910954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592fd61-893f-4bf4-9684-07d2f116cb94 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:03.915240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:03.915853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:03.917317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:03.917317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:03.927045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:03.928417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595fcc2-3cb2-43fe-930c-fad840362596 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:03.938264 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1548/3142] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:57:04.161152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:04.163203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:04.248905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d68f5c-adc9-43b5-b1ff-9e7b4fe66167 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:04.256164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:04.256904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:04.257297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:04.257714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:04.259650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:04.344217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:04.348132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:04.357667 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1595/3143] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:05.161796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa500f-b3d4-48f5-86aa-d0c46c45d711 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:05.162321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa500f-b3d4-48f5-86aa-d0c46c45d711 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:05.162693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa500f-b3d4-48f5-86aa-d0c46c45d711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:05.199015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a6fa500f-b3d4-48f5-86aa-d0c46c45d711 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:05.217895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a6fa500f-b3d4-48f5-86aa-d0c46c45d711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:05.285241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:05.352213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1549/3144] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:05.372530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:05.375816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:05.455793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:05.484484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fc8ac-97bf-4221-ae2c-edaa133b7fe3 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:05.494592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:05.495265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:05.496609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:05.497380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:05.509748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:05.511338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:05.530193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1596/3145] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:57:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:57:07.149747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:07.150342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:07.150342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:07.196209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:07.197717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:07.230143 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1550/3146] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:07.242056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:07.254390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ef060-80c9-4f61-a5e6-636ea7c87a12 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:07.269149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:07.270065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:07.270583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:07.271044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:07.287690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:07.289559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5f496-4551-43a6-a947-f8bd61ec247b service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:07.304332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1597/3147] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:57:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:57:08.200348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:08.206470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:08.435209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e24152-1c1e-4c58-8520-4948f2e2b9b3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:08.446745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:08.453750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:08.454639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:08.455169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:08.460640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:08.597166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:08.598767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1f9b6-c787-476f-8345-125bbaacab06 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:08.618080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1551/3148] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:08.664545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4467d-936e-494e-b48c-77cb979b34eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:08.665043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4467d-936e-494e-b48c-77cb979b34eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:08.665330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4467d-936e-494e-b48c-77cb979b34eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:08.746312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d9f4467d-936e-494e-b48c-77cb979b34eb None None] MFA Rules not processed for user `2f062663eef74d63aea291eaa5f213a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:08.747767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d9f4467d-936e-494e-b48c-77cb979b34eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:09.045473 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1598/3149] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:09.059575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:09.059975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:09.060676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:09.077831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:09.078472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:09.079447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:09.147862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] MFA Rules not processed for user `2f062663eef74d63aea291eaa5f213a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:09.151123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:09.167134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:09.170918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:09.193033 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1552/3150] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:09.200242 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1599/3151] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:57:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:09.219948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:09.226189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:09.253629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd71d4-fc03-4730-9c68-579deb43e92f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:09.263049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1fbba-08ad-4da5-a6eb-b02e0ee5e444 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:09.264411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:09.265537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:09.266020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:09.266788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:09.281164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:09.282615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:09.283247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:09.284234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:09.284809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:09.289101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1955207-e332-46d0-868c-62523f7a0dae service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:09.307255 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1553/3152] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:57:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:57:09.319656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:09.321376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:09.354281 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1600/3153] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:10.073025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491acb14-db30-4a9b-a46a-a300047298b3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:10.074159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491acb14-db30-4a9b-a46a-a300047298b3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:10.075238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491acb14-db30-4a9b-a46a-a300047298b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:10.120124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-491acb14-db30-4a9b-a46a-a300047298b3 None None] MFA Rules not processed for user `00585dbe7f814eeebda43417ca1d0ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:10.122989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-491acb14-db30-4a9b-a46a-a300047298b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:10.134726 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1554/3154] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:10.218730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:10.221828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:10.404601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06876-b146-490f-89c2-6010c7df4548 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:10.410685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:10.411857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:10.412365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:10.412920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] PATH_INFO: `/v3/users/9dd09a0889c44ce18f0c361c7c661465` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:10.414831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd09a0889c44ce18f0c361c7c661465)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:10.420269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:10.508216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:10.523238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd09a0889c44ce18f0c361c7c661465'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:10.525251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd09a0889c44ce18f0c361c7c661465'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:10.547044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd09a0889c44ce18f0c361c7c661465'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:10.567394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd09a0889c44ce18f0c361c7c661465'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:10.568205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd09a0889c44ce18f0c361c7c661465'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:10.588748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1601/3155] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:10 2022] DELETE /identity/v3/users/9dd09a0889c44ce18f0c361c7c661465 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:10.593912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449dd3e-e9fa-4b7e-a250-7b56fbcc2b1c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:10.594610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449dd3e-e9fa-4b7e-a250-7b56fbcc2b1c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:10.597154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449dd3e-e9fa-4b7e-a250-7b56fbcc2b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:10.624424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4449dd3e-e9fa-4b7e-a250-7b56fbcc2b1c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:10.625935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4449dd3e-e9fa-4b7e-a250-7b56fbcc2b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:10.711881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:10.767213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1555/3156] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:10.918103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:10.919600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:11.031223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891f075-adde-4846-9d7c-3daa33f77a53 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:11.038300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:11.039193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:11.039654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:11.040428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:11.056451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:11.058455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:11.069571 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1602/3157] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:11.442172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:11.442463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:11.442769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:11.447137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:11.448468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:11.488974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] MFA Rules not processed for user `00585dbe7f814eeebda43417ca1d0ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:11.490981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:11.664405 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1556/3158] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:11.709654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b6d35e-8ea2-4640-b3e1-caedfbe2d8da service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:11.723089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:11.723846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:11.724375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:11.724913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:11.731691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:11.884302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:11.912119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:11.913715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:11.938414 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1603/3159] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:57:11.939170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:11.941666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:11.989716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363af04-f6f8-4f51-ac15-85b527e948a3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:12.014435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:12.016341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:12.018690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:12.019149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] PATH_INFO: `/v3/projects/749824e2c16946a58801200784fa0480` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:12.023043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=749824e2c16946a58801200784fa0480)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:12.027939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:12.149210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749824e2c16946a58801200784fa0480', 'request_id': 'req-313dbf99-2c82-4048-b964-f205a5b64a4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.163369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749824e2c16946a58801200784fa0480', 'request_id': 'req-313dbf99-2c82-4048-b964-f205a5b64a4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.185743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749824e2c16946a58801200784fa0480', 'request_id': 'req-313dbf99-2c82-4048-b964-f205a5b64a4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.205383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] The token cache is being invalidate because project 749824e2c16946a58801200784fa0480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:12.206521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.207855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.210973 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1557/3160] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:11 2022] DELETE /identity/v3/projects/749824e2c16946a58801200784fa0480 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:12.218785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:12.251436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf02b27-b9b8-46d0-9c2f-44c1a010d377 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:12.464689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:12.467512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:12.469073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:12.471955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] PATH_INFO: `/v3/users/2f607c3c0fcf4e97b7208cb7034b0ec4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:12.478258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f607c3c0fcf4e97b7208cb7034b0ec4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:12.489327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:12.801863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f607c3c0fcf4e97b7208cb7034b0ec4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.805030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f607c3c0fcf4e97b7208cb7034b0ec4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.832840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f607c3c0fcf4e97b7208cb7034b0ec4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.878310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f607c3c0fcf4e97b7208cb7034b0ec4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.881034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f7989ceb-ddf3-4d34-8264-20d92536dc72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f607c3c0fcf4e97b7208cb7034b0ec4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:12.921173 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1604/3161] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:12 2022] DELETE /identity/v3/users/2f607c3c0fcf4e97b7208cb7034b0ec4 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:14.073793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:14.075736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:14.107831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dbf99-2c82-4048-b964-f205a5b64a4b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:14.228697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:14.229483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:14.229847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:14.230194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] PATH_INFO: `/v3/projects/419492810b3341d19f1f05c4ebed60cf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:14.231864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=419492810b3341d19f1f05c4ebed60cf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:14.257188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:14.281774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:14.410428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419492810b3341d19f1f05c4ebed60cf', 'request_id': 'req-a3beb381-8094-4a93-8cae-eeda6d878409'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:14.444920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419492810b3341d19f1f05c4ebed60cf', 'request_id': 'req-a3beb381-8094-4a93-8cae-eeda6d878409'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:14.469250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419492810b3341d19f1f05c4ebed60cf', 'request_id': 'req-a3beb381-8094-4a93-8cae-eeda6d878409'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:14.514824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] The token cache is being invalidate because project 419492810b3341d19f1f05c4ebed60cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:14.515491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:14.517049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3beb381-8094-4a93-8cae-eeda6d878409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:14.519881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1558/3162] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:14 2022] DELETE /identity/v3/projects/419492810b3341d19f1f05c4ebed60cf => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:14.528380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:14.529322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:14.529758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:14.575743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:14.577361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:14.694872 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1605/3163] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:14.703241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:14.704053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:14.704630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:14.741505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:14.743195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:14.757749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1559/3164] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:14.758528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:14.772204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8eb93-c1ef-4615-8815-fcb27eaf5148 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:14.784739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:14.785809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:14.786315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:14.786993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:14.788805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:57:14.789389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:14.791575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:14.803917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:14.818416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:14.819338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1606/3165] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:57:14.828161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8970838-83ad-4132-b404-ec94a94e0cbd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:14.833494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:14.834090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:14.834574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:14.835222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:14.835477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:14.838137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:14.873583 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1560/3166] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:14 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:14.874630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:14.894123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9774e6f-438c-4865-9ef8-f27cac471920 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:15.009176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:15.010103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.010452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.010830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.011652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:15.014797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:15.042654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1607/3167] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:14 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:15.046431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:15.059215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51a8c8-f731-4e9a-89ad-8ff3230fb82f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:15.065580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:15.066694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.067075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.067537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.068281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:15.070072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:15.085569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1561/3168] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:15.086369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:15.099034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6841fce-ce61-4d6f-a4a1-8987f0a07d97 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:15.106533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:15.107685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.108064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.108462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.109203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:15.110982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aefdfb-5df9-42b5-aa0f-7811ff9d4c53 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:15.130127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:15.137436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1608/3169] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:15.142176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa413b-2a5d-412b-8510-f041a43a5f6c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:15.148145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:15.148931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.149638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.150201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] PATH_INFO: `/v3/projects/c4004a1dbc2441f8a5c4ace42af3e2df/users/31761ade4b4a4df580fd5de55426eb0b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.171393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4004a1dbc2441f8a5c4ace42af3e2df, user_id=31761ade4b4a4df580fd5de55426eb0b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:15.176085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:15.215516 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1562/3170] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:15 2022] PUT /identity/v3/projects/c4004a1dbc2441f8a5c4ace42af3e2df/users/31761ade4b4a4df580fd5de55426eb0b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:15.223521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.223992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.225241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.276544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:15.277851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:15.367967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1609/3171] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:15.385092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:15.388422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:15.520386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf6aa-c523-41ff-b67b-3c1374097bab admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:15.536718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:15.537357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:15.537644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:15.537922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:15.567592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:15.572944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:15.605313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1563/3172] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:25.358070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:25.555651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:25.560588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:25.587706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:25.591290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5faa0-a738-4526-9a73-364e8ccae2fa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:25.604043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:25.604799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:25.605303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:25.605643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] PATH_INFO: `/v3/users/76b517a69e1f4dd0895104befebc128d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:25.612539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b517a69e1f4dd0895104befebc128d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:25.616830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:25.735705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b517a69e1f4dd0895104befebc128d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:25.736532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b517a69e1f4dd0895104befebc128d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:25.748399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b517a69e1f4dd0895104befebc128d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:25.767069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b517a69e1f4dd0895104befebc128d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:25.769235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d214f84-e3a9-4f4f-bcff-9e9159b7ac08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b517a69e1f4dd0895104befebc128d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:25.787397 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1610/3173] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:25 2022] DELETE /identity/v3/users/76b517a69e1f4dd0895104befebc128d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:26.958960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:26.973162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad364b25-ceeb-4604-aaec-c120af76610a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:27.129090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:27.130778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:27.131090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:27.131467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] PATH_INFO: `/v3/projects/cfa59dfe189849238f3693696fd1089e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:27.133065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa59dfe189849238f3693696fd1089e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:27.135025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:27.207151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa59dfe189849238f3693696fd1089e', 'request_id': 'req-0fb12093-06a9-4b33-90cf-e2b864668e1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:27.213162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa59dfe189849238f3693696fd1089e', 'request_id': 'req-0fb12093-06a9-4b33-90cf-e2b864668e1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:27.223940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa59dfe189849238f3693696fd1089e', 'request_id': 'req-0fb12093-06a9-4b33-90cf-e2b864668e1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:27.238514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] The token cache is being invalidate because project cfa59dfe189849238f3693696fd1089e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:27.239246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:27.240140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fb12093-06a9-4b33-90cf-e2b864668e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:27.242014 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1564/3174] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:26 2022] DELETE /identity/v3/projects/cfa59dfe189849238f3693696fd1089e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:27.251317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:27.258534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:27.259018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:27.286688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:27.293584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:27.499376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1611/3175] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:27.501157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:27.501456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:27.501744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:27.548779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:27.550678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:27.583044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:27.583501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1565/3176] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:27.609821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa04468-70b5-404a-ae41-5118ebcdd200 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:27.625662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:27.626353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:27.626731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:27.627686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:27.656385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:57:27.656715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:27.662455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:27.701468 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1612/3177] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:57:27.702902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:27.731470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94404f00-4388-4015-9b60-3fe494746a6b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:27.740721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:27.741494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:27.741873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:27.742288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:27.743065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:27.747329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:27.825875 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1566/3178] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:27 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:27.827271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:27.848803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b99ef-5754-4fa6-a8b8-d3f641d37200 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:28.011005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:28.011631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.011996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.012267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.012902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:28.022728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:28.034330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:28.069961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1613/3179] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:27 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:28.070213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:28.092976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0cf27-695f-4408-bf9f-f41b9b94acb4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:28.099331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:28.100053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.100401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.100740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.101468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:28.103261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:28.133530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1567/3180] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:28.134353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:28.149135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fa1f8-a1a3-44f3-8ddf-a98190ade898 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:28.156033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:28.157073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.157580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.158075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.160941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:28.163233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1997-9549-4104-a271-d7eec0f0969d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:28.182889 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1614/3181] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:28.185483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:28.207965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ff0cb-5204-4e3a-a154-0ec62f38ca1a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:28.213826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:28.215985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.216566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.217703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] PATH_INFO: `/v3/projects/ec085cc2148f4cda9483fbe7257eeb14/users/6a38dd114ec04ba5826a68a9e796ab8d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.250872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec085cc2148f4cda9483fbe7257eeb14, user_id=6a38dd114ec04ba5826a68a9e796ab8d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:28.256414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:28.321619 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1568/3182] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:28 2022] PUT /identity/v3/projects/ec085cc2148f4cda9483fbe7257eeb14/users/6a38dd114ec04ba5826a68a9e796ab8d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:28.327961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9a13-4b28-41bc-8424-addda64192cf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.329044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9a13-4b28-41bc-8424-addda64192cf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.329316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9a13-4b28-41bc-8424-addda64192cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.376085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ddbd9a13-4b28-41bc-8424-addda64192cf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:28.382486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ddbd9a13-4b28-41bc-8424-addda64192cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:28.554183 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1615/3183] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:28.571404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:28.575074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:28.649266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:28.727937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93e1a6-d559-4dce-87c8-a5e7712845f3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:28.732216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:28.732714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:28.733092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:28.733447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:28.760112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:28.766860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:28.788731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1569/3184] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:57:30.739333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:30.739897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:30.739897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:30.776459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] MFA Rules not processed for user `31761ade4b4a4df580fd5de55426eb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:30.777891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:30.922237 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1616/3185] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:30.927761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:30.929396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:30.952109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bf01-7e20-4b4c-a813-ae317d292902 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:30.964942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:30.965884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:30.966335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:30.966812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:30.968033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:30.973695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.029992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.030298 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1570/3186] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:30 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:31.042019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86485ad-0dd5-445c-908f-0885bd994317 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.121932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.122994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.123640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.123924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.126128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:31.131185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.154658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:31.175037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.176416 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1617/3187] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:31 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:57:31.190251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e05e6-0895-4e4e-a2b5-f1d4bd416973 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.204820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.205602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.205946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.206300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.207054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:31.208981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.230731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1571/3188] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:31.232026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.250340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6561-3134-47a3-b65f-679c43dfe720 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.256475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.257596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.258201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.258858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] PATH_INFO: `/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b/users/e963d81423304dadbbf445c18661e767/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.295415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f3f539c41f4979a3f8f7f268f5357b, user_id=e963d81423304dadbbf445c18661e767, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:31.298741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.331994 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1618/3189] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:31 2022] PUT /identity/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b/users/e963d81423304dadbbf445c18661e767/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:31.332399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.353221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837acf94-5a60-440e-ac96-01548220a57f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.436927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.437982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.438339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.438735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.439562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:31.441454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.470145 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1572/3190] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:31.474513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.490060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45218e-1031-4ccf-b427-640972528881 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.506224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.507931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.508590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.509261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] PATH_INFO: `/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b/users/e963d81423304dadbbf445c18661e767/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.514787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f3f539c41f4979a3f8f7f268f5357b, user_id=e963d81423304dadbbf445c18661e767, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:31.526696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:31.562111 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1619/3191] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:31 2022] PUT /identity/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b/users/e963d81423304dadbbf445c18661e767/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:31.569492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:31.571253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:31.820839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1aedd-a044-4626-9968-87e35c39ea1e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:31.864817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:31.975825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:31.977278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:31.977743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:31.978200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:31.980476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:32.000849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:32.003940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:32.017796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1573/3192] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:34.026921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:34.138454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353200f0-1ce4-45b3-9920-361c32bf8bc6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:34.150805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:34.151968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:34.152319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:34.152698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:34.155526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:34.298897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:34.301022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:34.324308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1620/3193] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:38.733836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:38.745065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:39.008365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c8818-f359-424e-aafc-10f2b9ec91b5 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:39.039879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:39.041215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:39.041790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:39.042389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] PATH_INFO: `/v3/users/56064ff9075243879d7bce37e3e547e7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:39.043841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56064ff9075243879d7bce37e3e547e7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:39.047791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:39.143963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56064ff9075243879d7bce37e3e547e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:39.144762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56064ff9075243879d7bce37e3e547e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:39.158438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56064ff9075243879d7bce37e3e547e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:39.172909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56064ff9075243879d7bce37e3e547e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:39.174280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56064ff9075243879d7bce37e3e547e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:39.213982 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1574/3194] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:38 2022] DELETE /identity/v3/users/56064ff9075243879d7bce37e3e547e7 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:39.741149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:39.742632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:39.758333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:39.759833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:39.771653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabcab8-5c98-4e2c-afc0-7a218e67e723 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:39.816938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:39.883963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf6ab5-7417-46a1-b62b-2d51a2f9b382 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:39.898403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:39.900084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:39.900848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:39.902048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:39.902778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:39.904079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:39.904442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:39.905755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] PATH_INFO: `/v3/projects/45916bd2a2e847b5ae05290dd106b04c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:39.909017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45916bd2a2e847b5ae05290dd106b04c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:39.911615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:39.918426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:39.949152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:39.953809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:39.982229 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1621/3195] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:40.046443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45916bd2a2e847b5ae05290dd106b04c', 'request_id': 'req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.063826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45916bd2a2e847b5ae05290dd106b04c', 'request_id': 'req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.091989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45916bd2a2e847b5ae05290dd106b04c', 'request_id': 'req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.116138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] The token cache is being invalidate because project 45916bd2a2e847b5ae05290dd106b04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:40.123430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.124942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.129814 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1575/3196] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:39 2022] DELETE /identity/v3/projects/45916bd2a2e847b5ae05290dd106b04c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:40.133789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:40.149973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15d62-3ca1-4fa3-82fc-e81c903588aa service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:40.178201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:40.314061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:40.316673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:40.318414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:40.318923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] PATH_INFO: `/v3/users/8c719e43f261418593e57d05bcff7701` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:40.323394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c719e43f261418593e57d05bcff7701)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:40.325577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:40.437821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c719e43f261418593e57d05bcff7701'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.440604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c719e43f261418593e57d05bcff7701'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.467836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c719e43f261418593e57d05bcff7701'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.496100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c719e43f261418593e57d05bcff7701'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.498795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-570bde86-fa2b-4d40-a50e-90df2236ea68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c719e43f261418593e57d05bcff7701'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:40.536142 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1622/3197] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:57:40 2022] DELETE /identity/v3/users/8c719e43f261418593e57d05bcff7701 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:41.200168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:41.204497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:41.245211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04787a-e8c5-42e6-b21a-8f48ad9b5449 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:41.388139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:41.389109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:41.389570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:41.390027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] PATH_INFO: `/v3/projects/86a66901b4074ab99c4f7acb86d25e08` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:41.391420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a66901b4074ab99c4f7acb86d25e08)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:41.394202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:41.518587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a66901b4074ab99c4f7acb86d25e08', 'request_id': 'req-692cb7f3-aa3b-4e1c-b681-8c3738517081'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:41.527062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:41.531604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a66901b4074ab99c4f7acb86d25e08', 'request_id': 'req-692cb7f3-aa3b-4e1c-b681-8c3738517081'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:41.552910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a66901b4074ab99c4f7acb86d25e08', 'request_id': 'req-692cb7f3-aa3b-4e1c-b681-8c3738517081'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:41.573552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] The token cache is being invalidate because project 86a66901b4074ab99c4f7acb86d25e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:57:41.573697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:41.574557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-692cb7f3-aa3b-4e1c-b681-8c3738517081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:57:41.576485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1576/3198] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:57:41 2022] DELETE /identity/v3/projects/86a66901b4074ab99c4f7acb86d25e08 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:41.582625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:41.583178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:41.583178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:41.626128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:41.627940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:41.775004 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1623/3199] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:41.784076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:41.784816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:41.785271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:41.818906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:41.821908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:41.843918 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1577/3200] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:41.847768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:41.858451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6e96b-170f-4bef-bdee-c83da7118925 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:41.881362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:41.882282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:41.882765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:41.883237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:41.885018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:57:41.885509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:41.887340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:41.920376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1624/3201] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:57:41.924906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:41.949437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ee7ba-6eea-4ef2-886d-4f6e897fcbd8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:41.963432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:41.964064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:41.964345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:41.964617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:41.965448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:41.969209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.028505 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1578/3202] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:57:41 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:42.032580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:42.044715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68590eea-c26b-4799-b76e-f1c541a396ea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:42.068536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:42.146884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:42.149225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.149518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.149799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.151614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:42.159187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.207944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:42.209116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1625/3203] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:57:42 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:57:42.224332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537885e-ba13-481d-86d2-5f1a6b0c238c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:42.237405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:42.238423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.238944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.239407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.240262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:42.243653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.269591 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1579/3204] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:42.271594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:42.290292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ccc-35b9-458e-b0be-88aecdaa527c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:42.305232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:42.305861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.306193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.306473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.307151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:42.308823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5353615-a90c-4aab-98e3-cbb47215e211 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.339189 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1626/3205] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:57:42.343483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:42.359313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1d55-5a14-4d96-909c-0a34fa829792 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:42.371065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:42.372609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.373054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.373489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] PATH_INFO: `/v3/projects/8c875b2746384d5a9d02548faab6fd5d/users/c84e7493d0ed47aca659dc925e8ca39c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.424204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c875b2746384d5a9d02548faab6fd5d, user_id=c84e7493d0ed47aca659dc925e8ca39c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:42.431294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.459182 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1580/3206] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:57:42 2022] PUT /identity/v3/projects/8c875b2746384d5a9d02548faab6fd5d/users/c84e7493d0ed47aca659dc925e8ca39c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:57:42.461486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06192-193f-48b4-8c53-613c294205c5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.462004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06192-193f-48b4-8c53-613c294205c5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.463031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06192-193f-48b4-8c53-613c294205c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.502816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-feb06192-193f-48b4-8c53-613c294205c5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:42.510807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-feb06192-193f-48b4-8c53-613c294205c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:42.669364 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1627/3207] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:42.709782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:42.711891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:42.874291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286073-0e26-4f32-badf-5dfa99c4ea0c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:42.885202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:42.885880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:42.886157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:42.886423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:42.930167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:42.932005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:42.960676 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1581/3208] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:43.489304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:43.489960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:43.490426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:43.575892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] MFA Rules not processed for user `6a38dd114ec04ba5826a68a9e796ab8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:43.577233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:43.741873 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1628/3209] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:43.911782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:43.914392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:43.933794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15e617-82b2-41a2-93bc-88b10e33e939 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:43.949512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:43.950381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:43.950851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:43.951313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:43.990377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:43.995880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73e22d-a980-4a31-9aeb-a5806d59d532 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:44.018353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1582/3210] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:46.385507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:46.387406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:46.430520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:46.489546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c3a8d-3f8c-40ba-9a93-a6f162a81abb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:46.499463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:46.500452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:46.500717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:46.501093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:46.504346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:46.519610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:46.521247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:46.534761 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1629/3211] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:57:48.365671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4411-f038-4cbd-bc61-a87792139a36 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:48.366147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4411-f038-4cbd-bc61-a87792139a36 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:48.366240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d4411-f038-4cbd-bc61-a87792139a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:48.508083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bc0d4411-f038-4cbd-bc61-a87792139a36 None None] MFA Rules not processed for user `e963d81423304dadbbf445c18661e767`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:48.511989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bc0d4411-f038-4cbd-bc61-a87792139a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:48.719723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:48.756901 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1583/3212] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:48.764015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:48.766080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:48.807958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2291-2573-4ca1-b550-52142c3b1ac8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:48.808574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2291-2573-4ca1-b550-52142c3b1ac8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:48.809951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a2291-2573-4ca1-b550-52142c3b1ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:48.873932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-818a2291-2573-4ca1-b550-52142c3b1ac8 None None] MFA Rules not processed for user `6a38dd114ec04ba5826a68a9e796ab8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:48.879972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-818a2291-2573-4ca1-b550-52142c3b1ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:48.913208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1584/3213] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:57:48.956914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9955-712b-4c29-bf2f-e3d2a94644e5 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:48.964770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:48.965803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:48.966447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:48.967020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:48.969531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:49.078117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:49.080795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:49.108119 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1630/3214] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:57:52.264973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f0f0-e868-4b4e-92b8-e6c4616b2b59 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:52.265459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f0f0-e868-4b4e-92b8-e6c4616b2b59 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:52.265562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258f0f0-e868-4b4e-92b8-e6c4616b2b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:52.299354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0258f0f0-e868-4b4e-92b8-e6c4616b2b59 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:52.301556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0258f0f0-e868-4b4e-92b8-e6c4616b2b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:52.556254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1585/3215] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:57:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:52.591123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:52.596292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:52.776162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:57:52.824661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c78527-63af-480c-b3c6-204c64c176ad service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:52.856525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:52.857423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:52.857888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:52.858327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:52.880422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:52.882035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ed234-516f-44a6-addd-4830cf49b743 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:52.902684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1631/3216] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:57:58.534211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:58.534211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:58.534211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:58.632741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] MFA Rules not processed for user `c84e7493d0ed47aca659dc925e8ca39c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:58.634003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:58.906720 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1586/3217] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:58.909236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623784-dfca-453c-a8a0-40d79192f123 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:58.911329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623784-dfca-453c-a8a0-40d79192f123 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:58.911812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623784-dfca-453c-a8a0-40d79192f123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:58.951134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-44623784-dfca-453c-a8a0-40d79192f123 None None] MFA Rules not processed for user `c84e7493d0ed47aca659dc925e8ca39c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:58.952612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-44623784-dfca-453c-a8a0-40d79192f123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:58.973727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1632/3218] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:59.009353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:57:59.011299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:59.049078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3116d-f606-4c36-ae4c-2c1c43b0defc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:57:59.066221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:59.066593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:59.066809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:59.073516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:57:59.074147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:57:59.074422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:57:59.075944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:57:59.092494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:57:59.093732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] MFA Rules not processed for user `31761ade4b4a4df580fd5de55426eb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:57:59.094113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:57:59.095783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:57:59.112014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1633/3219] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:57:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:57:59.116674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1587/3220] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:57:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:01.166358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:01.181184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e746-49ed-4791-87a6-92b8704ef113 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:01.186685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:01.195113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:01.195408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:01.195689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:01.198339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:01.239077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:01.240816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:01.269760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1634/3221] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:08.399490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:08.401101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:08.444367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365cc-231a-4600-9f50-03df28f7e64e service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:08.472130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:08.478964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:08.479827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:08.482203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] PATH_INFO: `/v3/users/2f062663eef74d63aea291eaa5f213a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:08.485534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f062663eef74d63aea291eaa5f213a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:08.487791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:08.740170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f062663eef74d63aea291eaa5f213a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:08.740984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f062663eef74d63aea291eaa5f213a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:08.762389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f062663eef74d63aea291eaa5f213a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:08.775687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f062663eef74d63aea291eaa5f213a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:08.776767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1ac5989f-69f0-47ac-a5e4-bdcfcd980430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f062663eef74d63aea291eaa5f213a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:08.779263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:08.811956 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1588/3222] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:58:08 2022] DELETE /identity/v3/users/2f062663eef74d63aea291eaa5f213a4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:10.478938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:10.500699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830c3d8-6c68-4e38-bba3-920b326fa5f4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:10.726075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:10.726968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:10.727391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:10.727833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] PATH_INFO: `/v3/projects/7588851b153c4c0dbc7e63cfa1778497` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:10.729160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7588851b153c4c0dbc7e63cfa1778497)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:10.732467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:10.868704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7588851b153c4c0dbc7e63cfa1778497', 'request_id': 'req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:10.878388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7588851b153c4c0dbc7e63cfa1778497', 'request_id': 'req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:10.903968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7588851b153c4c0dbc7e63cfa1778497', 'request_id': 'req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:10.970804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] The token cache is being invalidate because project 7588851b153c4c0dbc7e63cfa1778497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:58:10.971647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:10.976898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8e54088e-4d6d-47c5-a5a0-f9647f2948e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:10.994439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1635/3223] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:58:10 2022] DELETE /identity/v3/projects/7588851b153c4c0dbc7e63cfa1778497 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:11.000721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5756a-757b-496a-b093-757f9b736eac None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.001167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5756a-757b-496a-b093-757f9b736eac None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.001274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5756a-757b-496a-b093-757f9b736eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.077060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-31a5756a-757b-496a-b093-757f9b736eac None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:11.078312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-31a5756a-757b-496a-b093-757f9b736eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:11.165675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1589/3224] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:11.173074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.175036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.175036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.222251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:11.228260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:11.262725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1636/3225] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:11.266791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5756a-757b-496a-b093-757f9b736eac None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.303331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5756a-757b-496a-b093-757f9b736eac None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.313811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.315408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.316284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.319797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.320691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:58:11.322999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.327472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.356856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1590/3226] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:58:11.362425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.406837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dabdc1-50c2-4556-b200-1d123c594ec1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.425930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.426589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.426883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.427151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.427793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.429731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.492108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1637/3227] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:58:11 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:11.496386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.513662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21521a8-a60b-4172-9e7b-dd760aa0e082 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.623857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.625660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:11.676455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.677446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.677973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.678442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.679464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.682312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.706016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:11.751233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1591/3228] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:58:11 2022] POST /identity/v3/users => generated 290 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:58:11.753059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.784932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1451a1-62ec-4f4e-82de-374fb581456a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.798900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.801389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.804481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.805311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.807381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.813627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.843717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b4a89-fe0b-4ad7-ba61-c0765a2a33bc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.846273 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1592/3229] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:11.849108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.859181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.859971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.863395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.867560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] PATH_INFO: `/v3/users/9c03c67e0c4c4de7a186f28ac10d0b92` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.870431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c03c67e0c4c4de7a186f28ac10d0b92)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.876274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.886664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205487ea-c7d4-41e2-9795-3b038380f16e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.900231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.902396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.902809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.905818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:11.907527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:11.911544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:11.951790 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1593/3230] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:11.956323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:11.976706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4127fc9-be55-437b-9b8e-c52369b37f95 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:11.988320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:11.989212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:11.989605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:11.990658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] PATH_INFO: `/v3/projects/c1e5d6f3d41e4aa49e630f8c7ac87ded/users/6493da825aa24f43bdf8f0835a16e3bb/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:12.023714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e5d6f3d41e4aa49e630f8c7ac87ded, user_id=6493da825aa24f43bdf8f0835a16e3bb, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:12.027304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec44ea-dbf7-4188-8cc7-df00025d3eba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:12.052808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c03c67e0c4c4de7a186f28ac10d0b92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:12.060896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1594/3231] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:58:11 2022] PUT /identity/v3/projects/c1e5d6f3d41e4aa49e630f8c7ac87ded/users/6493da825aa24f43bdf8f0835a16e3bb/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:12.063000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c03c67e0c4c4de7a186f28ac10d0b92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:12.066532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:12.066972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:12.067361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:12.091413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c03c67e0c4c4de7a186f28ac10d0b92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:12.127879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c03c67e0c4c4de7a186f28ac10d0b92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:12.131232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c03c67e0c4c4de7a186f28ac10d0b92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:12.145094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:12.146692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:12.158587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:12.172416 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1638/3232] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:58:11 2022] DELETE /identity/v3/users/9c03c67e0c4c4de7a186f28ac10d0b92 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:58:12.292838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1595/3233] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:58:12.321116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:12.322599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:12.478682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77871e-78e6-4925-8022-1609ad3b24d9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:12.478682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:12.478682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:12.478682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:12.479819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:12.557835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:12.595258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:12.599621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:12.614928 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1639/3234] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:12.690781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:12.692698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:12.719209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907248-9a1e-4a76-a880-dc21df2ee355 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:12.806013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:12.820360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:12.828615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:12.834965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] PATH_INFO: `/v3/projects/a2b7e30f79bf42e497c53c091ad0a106` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:12.876406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b7e30f79bf42e497c53c091ad0a106)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:12.899593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:13.091707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b7e30f79bf42e497c53c091ad0a106', 'request_id': 'req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.187157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b7e30f79bf42e497c53c091ad0a106', 'request_id': 'req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.222704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b7e30f79bf42e497c53c091ad0a106', 'request_id': 'req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.281381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] The token cache is being invalidate because project a2b7e30f79bf42e497c53c091ad0a106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:58:13.281687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.284845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.293063 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1596/3235] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:58:12 2022] DELETE /identity/v3/projects/a2b7e30f79bf42e497c53c091ad0a106 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:13.297311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:13.321052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5a02-3cea-4358-8cd2-b2b4b7f17d10 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:13.527192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:13.528817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:13.529188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:13.529475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] PATH_INFO: `/v3/users/68910ac4980a4f74895c0a5ab25199c9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:13.531530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] RBAC: Authorizing `identity:delete_user(user_id=68910ac4980a4f74895c0a5ab25199c9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:13.533554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:13.683679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68910ac4980a4f74895c0a5ab25199c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.685913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68910ac4980a4f74895c0a5ab25199c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.697518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68910ac4980a4f74895c0a5ab25199c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.727540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68910ac4980a4f74895c0a5ab25199c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.728688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98c3aded-4f03-408d-bc2d-13ea194b1aac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68910ac4980a4f74895c0a5ab25199c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:13.761328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1640/3236] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:58:13 2022] DELETE /identity/v3/users/68910ac4980a4f74895c0a5ab25199c9 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:14.543461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:14.546253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:14.596413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300871e5-ab5d-43ba-bf4e-00c05dda9c9c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:14.830464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:14.831308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:14.831699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:14.832054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] PATH_INFO: `/v3/projects/5d5c7562e0a14c8997a94b916a0871de` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:14.834437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5c7562e0a14c8997a94b916a0871de)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:14.836565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:14.949735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:14.971391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5c7562e0a14c8997a94b916a0871de', 'request_id': 'req-b426c19a-c4e7-4249-83c2-977db6ff6707'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:14.983360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5c7562e0a14c8997a94b916a0871de', 'request_id': 'req-b426c19a-c4e7-4249-83c2-977db6ff6707'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:15.036076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5c7562e0a14c8997a94b916a0871de', 'request_id': 'req-b426c19a-c4e7-4249-83c2-977db6ff6707'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:15.070418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] The token cache is being invalidate because project 5d5c7562e0a14c8997a94b916a0871de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:58:15.070916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:15.075051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b426c19a-c4e7-4249-83c2-977db6ff6707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:15.081856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1597/3237] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:58:14 2022] DELETE /identity/v3/projects/5d5c7562e0a14c8997a94b916a0871de => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:15.083287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.083757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.083833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.136034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:15.137291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:15.255641 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1641/3238] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:15.257473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.257866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.258222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.305311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:15.310721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:15.332451 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1598/3239] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:15.337579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.353184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad73b-37b6-4ee9-a06c-9d08226caf65 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.362909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.363841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.364420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.364874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.365798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:58:15.366280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.368595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.373460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:15.391723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1642/3240] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:58:15.391947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.404889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b589c91-eaff-45e6-a8d5-05b144c97e3c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.419818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.420728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.421129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.421551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.422276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.426976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.477535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1599/3241] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:58:15 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:15.479169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.496407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fb2e8-cabe-4b87-870c-a33b6216faef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.620411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.621042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.621324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.621598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.622235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.624206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.684970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1643/3242] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:58:15 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:15.686688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.740578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72676d26-835f-4935-81e5-cc03bbf7ae5c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.772097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.773566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.774205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.775284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.776782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.779549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.811693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1600/3243] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:15.816681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.832297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0127d-eb42-48b5-b240-863c393ce51b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.836458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.837626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.838056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.838334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.839712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.841189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fba79-755b-4848-ae32-a63962627534 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.865080 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1644/3244] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:15.869168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:15.882127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbdd13-a092-4ebf-be4e-ddcc9303ac8f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:15.890524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:15.891342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.891725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.892628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] PATH_INFO: `/v3/projects/ffbf416811f64df0b56656a0cf7778ce/users/1a343dddda044b8fa4eb797a4b64556a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:15.932457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbf416811f64df0b56656a0cf7778ce, user_id=1a343dddda044b8fa4eb797a4b64556a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:15.935344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:15.963576 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1601/3245] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:58:15 2022] PUT /identity/v3/projects/ffbf416811f64df0b56656a0cf7778ce/users/1a343dddda044b8fa4eb797a4b64556a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:15.979692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:15.980401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:15.980945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:16.042359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] MFA Rules not processed for user `1a343dddda044b8fa4eb797a4b64556a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:16.044729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:16.197602 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1645/3246] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:16.199509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:16.406516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd5de8-d9ab-4114-a8f1-e16cbfb37f69 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:16.418768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:16.419364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:16.419640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:16.420978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:16.420978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:16.422469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:16.482407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1602/3247] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:58:16 2022] POST /identity/v3/projects => generated 387 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:16.486308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:16.508071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f89461-66e0-4808-92df-43b78cb2f69e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:16.699161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:16.699809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:16.700122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:16.700373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:16.701049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:16.705063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:16.734639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:16.752384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1646/3248] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:58:16 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:16.756415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:16.771718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ecc81-d617-49fe-8a59-4df7cf1f6420 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:16.796603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:16.797231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:16.797511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:16.797786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:16.799219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:16.806651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:16.844116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:16.854886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1603/3249] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:16.860152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4294-6102-46c8-802b-9ae3d8b2d7d6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:16.885504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:16.886194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:16.886477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:16.886797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] PATH_INFO: `/v3/projects/f3cec66ec0674490b3eaa3f40338f61e/users/b5dab333d06f4c4c997e9b771a72e269/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:16.943193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cec66ec0674490b3eaa3f40338f61e, user_id=b5dab333d06f4c4c997e9b771a72e269, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:16.951333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:16.989278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:16.989749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1647/3250] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:58:16 2022] PUT /identity/v3/projects/f3cec66ec0674490b3eaa3f40338f61e/users/b5dab333d06f4c4c997e9b771a72e269/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:58:17.009028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ec699-b49b-48b2-bb97-da16f328586f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:17.128978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:17.191171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:17.191797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:17.192074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:17.192346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:17.193010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:17.194686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f67eb-ecee-4068-9bcd-8b40baf9b0ca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:17.220380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:17.230820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1604/3251] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:17.233096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6127f-57bb-44ed-8dfb-b0aee26f1682 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:17.253331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:17.254024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:17.254310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:17.254626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] PATH_INFO: `/v3/projects/f3cec66ec0674490b3eaa3f40338f61e/users/b5dab333d06f4c4c997e9b771a72e269/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:17.258454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cec66ec0674490b3eaa3f40338f61e, user_id=b5dab333d06f4c4c997e9b771a72e269, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:17.261320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7338e-b216-476e-b185-dfe42e7e3c5c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:17.294089 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1648/3252] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:58:17 2022] PUT /identity/v3/projects/f3cec66ec0674490b3eaa3f40338f61e/users/b5dab333d06f4c4c997e9b771a72e269/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:58:17.305678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb55c-c549-401d-9628-f03067e948cc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:17.306158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb55c-c549-401d-9628-f03067e948cc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:17.306661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bb55c-c549-401d-9628-f03067e948cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:17.341995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-369bb55c-c549-401d-9628-f03067e948cc None None] MFA Rules not processed for user `b5dab333d06f4c4c997e9b771a72e269`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:17.359886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-369bb55c-c549-401d-9628-f03067e948cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:17.491466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:17.491761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:17.492029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:17.495891 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1605/3253] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:17.517324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] MFA Rules not processed for user `1a343dddda044b8fa4eb797a4b64556a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:17.520743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:17.664375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1649/3254] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:17.677634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb55c-c549-401d-9628-f03067e948cc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:17.679274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-369bb55c-c549-401d-9628-f03067e948cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:18.045895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bb55c-c549-401d-9628-f03067e948cc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:18.059387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:18.061206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:18.061499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:18.061774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:18.107370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:18.114052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:18.156625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1606/3255] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:24.300617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:24.302812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:24.393396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855f033-cdfc-4473-80dc-8686c180bc14 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:24.398669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:24.399552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:24.399977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:24.400379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] PATH_INFO: `/v3/users/3aa253ded2304ac3917a1788b6a1500b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:24.401694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa253ded2304ac3917a1788b6a1500b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:24.403677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:24.464049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa253ded2304ac3917a1788b6a1500b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.464931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa253ded2304ac3917a1788b6a1500b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.482583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa253ded2304ac3917a1788b6a1500b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.502243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:24.512338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa253ded2304ac3917a1788b6a1500b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.513439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41b10d16-6435-4c5a-a72f-12417af4e28c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa253ded2304ac3917a1788b6a1500b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.531884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1650/3256] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:58:24 2022] DELETE /identity/v3/users/3aa253ded2304ac3917a1788b6a1500b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:24.745740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:24.769982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b66d-b065-4f4f-b85d-4088fa17ebb8 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:24.791758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:24.859217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:24.859851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:24.860125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:24.860392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] PATH_INFO: `/v3/projects/71f6b606beca4eb5b62c89a945be15c7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:24.862885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f6b606beca4eb5b62c89a945be15c7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:24.865214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:24.925516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f6b606beca4eb5b62c89a945be15c7', 'request_id': 'req-8d03a55e-245c-438f-a83c-cdb992a448d6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.934903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f6b606beca4eb5b62c89a945be15c7', 'request_id': 'req-8d03a55e-245c-438f-a83c-cdb992a448d6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.944919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f6b606beca4eb5b62c89a945be15c7', 'request_id': 'req-8d03a55e-245c-438f-a83c-cdb992a448d6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.964819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] The token cache is being invalidate because project 71f6b606beca4eb5b62c89a945be15c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:58:24.965106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.965905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d03a55e-245c-438f-a83c-cdb992a448d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:58:24.973154 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1607/3257] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:58:24 2022] DELETE /identity/v3/projects/71f6b606beca4eb5b62c89a945be15c7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:58:24.974910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:24.975238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:24.975526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:24.999812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:25.001830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:25.113349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1651/3258] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:25.119994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.120277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.120553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.165888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:25.167539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:25.178279 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1608/3259] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:25.182897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.201535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be909bf2-33c2-4472-83ba-e2de771880d3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.213410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.214324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.214648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.214930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.219686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:58:25.220796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.230971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.252878 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1652/3260] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:58:25.254301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.266088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea654d9-ea1c-4f3b-8a39-f3d2036a3d7a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.280690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.281266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.281540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.282264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.282541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.284983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.332562 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1609/3261] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:58:25 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:25.337321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.349998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76152c07-9d95-4318-ab73-9c1db9d7915b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.484781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.486195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.496630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.497463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.498445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.500979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.568052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1653/3262] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:58:25 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:58:25.570341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.587150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c0ab-328e-4944-ae8e-643827501330 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.595915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.597312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.597312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.597312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.598466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.600268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.618075 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1610/3263] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:25.619603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.635640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66395720-59a5-48ba-a567-acdcae52a69a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.644217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.658728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.660077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.662206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.666434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.669342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e59e0-fdd8-4995-8569-21a468aaf9ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.685131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1654/3264] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:58:25.689946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.701817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd86568-0269-4b5f-93f1-1e130886e42d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.708224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.708909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.709187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.709463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] PATH_INFO: `/v3/projects/8340f7f8fa9848dc822903c738f78f2a/users/ee21da3ff38a43798bce50c9aba4ae33/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.726102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8340f7f8fa9848dc822903c738f78f2a, user_id=ee21da3ff38a43798bce50c9aba4ae33, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.728869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.745212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1611/3265] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:58:25 2022] PUT /identity/v3/projects/8340f7f8fa9848dc822903c738f78f2a/users/ee21da3ff38a43798bce50c9aba4ae33/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:58:25.751922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0879de-abc5-4fa9-99e6-4e90f5f38663 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.752384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0879de-abc5-4fa9-99e6-4e90f5f38663 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.752702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0879de-abc5-4fa9-99e6-4e90f5f38663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.777338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1e0879de-abc5-4fa9-99e6-4e90f5f38663 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:25.778596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e0879de-abc5-4fa9-99e6-4e90f5f38663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:25.825942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:25.851210 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1655/3266] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:25.858249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:25.860003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:25.918602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:25.928750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c02343-1495-47f7-b053-04290a48f256 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:25.932433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:25.933059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:25.933338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:25.933660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:25.944080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:25.945523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:25.955948 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1612/3267] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:26.325663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1d34-13a9-46c4-902e-45795b44f6dd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:26.325954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1d34-13a9-46c4-902e-45795b44f6dd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:26.327115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae1d34-13a9-46c4-902e-45795b44f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:26.364283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4dae1d34-13a9-46c4-902e-45795b44f6dd None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:26.365526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4dae1d34-13a9-46c4-902e-45795b44f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:26.531037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1656/3268] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:26.546681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:26.548830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:26.665750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a7c75-f36e-42cc-af17-fa8dfb4ca011 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:26.672032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:26.672755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:26.673261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:26.673791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:26.683056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:26.684800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:26.708522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1613/3269] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 20:58:28.895990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0891d1-247d-4558-9a44-600c4c83b2fc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:28.897597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0891d1-247d-4558-9a44-600c4c83b2fc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:28.897875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0891d1-247d-4558-9a44-600c4c83b2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:28.974977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3e0891d1-247d-4558-9a44-600c4c83b2fc None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:28.977600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3e0891d1-247d-4558-9a44-600c4c83b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:29.195969 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1657/3270] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:58:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:58:29.204738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:29.206895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:29.396236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d589e3e-bbdb-466c-8fb1-31a543fec401 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:29.405230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:29.406686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:29.407215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:29.415129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:29.449766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:29.452327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2db8-2937-4684-9ad3-7523624293be service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:29.481734 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1614/3271] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:58:32.086680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cb21-6447-4fbc-b011-247903ce938f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:32.087136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cb21-6447-4fbc-b011-247903ce938f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:32.087955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cb21-6447-4fbc-b011-247903ce938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:32.140229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-18b4cb21-6447-4fbc-b011-247903ce938f None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:32.143858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-18b4cb21-6447-4fbc-b011-247903ce938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:32.164078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1658/3272] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:38.832023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:38.833196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:38.833673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:38.915689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:38.920682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:39.105487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:39.153302 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1615/3273] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:39.164809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:39.165348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:39.165348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:39.218979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:39.221223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:39.287660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1659/3274] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:39.549620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:39.572745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a68af6-e706-4852-a47f-6a15f7e2efd0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:39.585480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:39.586107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:39.586378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:39.586685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:39.611276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:39.614066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15082004-3dfc-48fb-b542-d3bb906007e0 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:39.639191 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1616/3275] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:47.966842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:58:47.989204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e885b3-57b8-4df4-a55a-68acf351d782 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:58:47.998194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:58:47.999531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:48.000609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:48.002564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:48.004812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:48.112575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:58:48.114450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a8492-eef1-4366-8b61-7fcf235cb67a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:58:48.125810 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1660/3276] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:58:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:58:48.603508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:58:48.603508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:58:48.603960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:58:48.636793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] MFA Rules not processed for user `1a343dddda044b8fa4eb797a4b64556a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:58:48.639832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:58:48.729921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1617/3277] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:58:58.120225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:58:58.721248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=35 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:00.289818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4762c-11ca-4122-8893-4e4173bae87c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:00.290258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4762c-11ca-4122-8893-4e4173bae87c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:00.290379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4762c-11ca-4122-8893-4e4173bae87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:00.327641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-68a4762c-11ca-4122-8893-4e4173bae87c None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:00.328977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68a4762c-11ca-4122-8893-4e4173bae87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:00.338110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1661/3278] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 20:59:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:00.366020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:00.380969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83256d84-303c-49cf-8b51-d800b7317e63 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:00.387093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:00.387635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:00.387917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:00.388205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:00.398624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:00.400613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae4bdd-de02-473c-9701-41de1759b7c9 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:00.411681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1618/3279] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:01.959723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabfdb-71ad-44c8-9cf6-d8a26d569d9b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:01.960194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabfdb-71ad-44c8-9cf6-d8a26d569d9b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:01.960278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabfdb-71ad-44c8-9cf6-d8a26d569d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:01.985511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-13cabfdb-71ad-44c8-9cf6-d8a26d569d9b None None] MFA Rules not processed for user `6a38dd114ec04ba5826a68a9e796ab8d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:01.987062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-13cabfdb-71ad-44c8-9cf6-d8a26d569d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:02.091575 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1662/3280] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:03.578465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e98a6d-fa01-4b30-bf18-9c2d65013781 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:03.579594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e98a6d-fa01-4b30-bf18-9c2d65013781 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:03.579594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e98a6d-fa01-4b30-bf18-9c2d65013781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:03.650558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4e98a6d-fa01-4b30-bf18-9c2d65013781 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:03.658349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4e98a6d-fa01-4b30-bf18-9c2d65013781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:03.682488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1619/3281] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:12.078252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=28 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:13.630376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:15.759600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:15.760186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:15.760404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:15.787215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] MFA Rules not processed for user `31761ade4b4a4df580fd5de55426eb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:15.788671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:15.883511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1663/3282] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:16.752836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74bf7-0868-43ec-a6fc-3e21d872d885 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:16.754065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74bf7-0868-43ec-a6fc-3e21d872d885 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:16.754590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74bf7-0868-43ec-a6fc-3e21d872d885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:16.807680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-18c74bf7-0868-43ec-a6fc-3e21d872d885 None None] MFA Rules not processed for user `c84e7493d0ed47aca659dc925e8ca39c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:16.808938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-18c74bf7-0868-43ec-a6fc-3e21d872d885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:16.928425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1620/3283] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:17.773245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:17.776485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:17.803694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777b899-844d-48a9-bef0-f5e93ba29689 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:17.809289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:17.810304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:17.810815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:17.811313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] PATH_INFO: `/v3/users/1a343dddda044b8fa4eb797a4b64556a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:17.814983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a343dddda044b8fa4eb797a4b64556a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:17.817163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:17.913797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a343dddda044b8fa4eb797a4b64556a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:17.914610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a343dddda044b8fa4eb797a4b64556a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:17.924718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a343dddda044b8fa4eb797a4b64556a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:17.952452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a343dddda044b8fa4eb797a4b64556a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:17.954178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a343dddda044b8fa4eb797a4b64556a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:17.998785 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1664/3284] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:17 2022] DELETE /identity/v3/users/1a343dddda044b8fa4eb797a4b64556a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:18.004565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:18.005673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:18.006112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:18.057498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:18.058864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:18.174986 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1621/3285] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:18.187199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:18.189528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:18.305546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c7fa9-d202-4777-846b-af6352fe2e20 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:18.311332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:18.312206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:18.313490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:18.313952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:18.330118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:18.332607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:18.345735 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1665/3286] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:18.999483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:19.001515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:19.024428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697721c-b3f1-426d-aa52-c3292e95837f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:19.035661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:19.037232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:19.037789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:19.038231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] PATH_INFO: `/v3/projects/ffbf416811f64df0b56656a0cf7778ce` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:19.040090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbf416811f64df0b56656a0cf7778ce)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:19.042183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:19.108660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbf416811f64df0b56656a0cf7778ce', 'request_id': 'req-7f98aee0-15ca-494f-ac00-d40c431fd22f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.116361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbf416811f64df0b56656a0cf7778ce', 'request_id': 'req-7f98aee0-15ca-494f-ac00-d40c431fd22f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.126231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbf416811f64df0b56656a0cf7778ce', 'request_id': 'req-7f98aee0-15ca-494f-ac00-d40c431fd22f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.141753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] The token cache is being invalidate because project ffbf416811f64df0b56656a0cf7778ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:19.142048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.143880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.146868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1622/3287] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:18 2022] DELETE /identity/v3/projects/ffbf416811f64df0b56656a0cf7778ce => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:19.150985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:19.163944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2a47d-5bf1-4f59-ab1a-b9f0cef5a66d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:19.187249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:19.241060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:19.242275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:19.242863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:19.243591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] PATH_INFO: `/v3/users/b5dab333d06f4c4c997e9b771a72e269` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:19.247186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dab333d06f4c4c997e9b771a72e269)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:19.248995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:19.346374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dab333d06f4c4c997e9b771a72e269'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.347107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dab333d06f4c4c997e9b771a72e269'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.355357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dab333d06f4c4c997e9b771a72e269'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.389184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dab333d06f4c4c997e9b771a72e269'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.390045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9d6a462b-95cd-43b0-bdcf-66b8e2b77f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dab333d06f4c4c997e9b771a72e269'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:19.414546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1666/3288] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:19 2022] DELETE /identity/v3/users/b5dab333d06f4c4c997e9b771a72e269 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:19.894264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:19.896811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:19.947632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98aee0-15ca-494f-ac00-d40c431fd22f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:20.037160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:20.088722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:20.089743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:20.090480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:20.091508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] PATH_INFO: `/v3/projects/f3cec66ec0674490b3eaa3f40338f61e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:20.094040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cec66ec0674490b3eaa3f40338f61e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:20.097710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:20.201458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cec66ec0674490b3eaa3f40338f61e', 'request_id': 'req-bd361b10-300f-434b-bab1-fa2e103cbf1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:20.220799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cec66ec0674490b3eaa3f40338f61e', 'request_id': 'req-bd361b10-300f-434b-bab1-fa2e103cbf1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:20.235753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cec66ec0674490b3eaa3f40338f61e', 'request_id': 'req-bd361b10-300f-434b-bab1-fa2e103cbf1d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:20.257895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] The token cache is being invalidate because project f3cec66ec0674490b3eaa3f40338f61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:20.258385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:20.272814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd361b10-300f-434b-bab1-fa2e103cbf1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:20.277447 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1623/3289] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:19 2022] DELETE /identity/v3/projects/f3cec66ec0674490b3eaa3f40338f61e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:20.284632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:20.285362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:20.285362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:20.335055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:20.338275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:20.482343 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1667/3290] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:20.491299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:20.492228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:20.492773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:20.538819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:20.540583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:20.558816 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1624/3291] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:59:20.563774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:20.580291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f283ea-baf1-431c-b1cc-aa22a40dc366 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:20.591430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:20.601143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:20.601828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:20.602239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:20.604217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:59:20.605257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:20.609931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:20.664932 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1668/3292] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:59:20.670141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:20.699165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb40b04-a616-4d92-ac9c-053aa3347766 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:20.731502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:20.731502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:20.735515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:20.735889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:20.738469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:20.745763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:20.855973 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1625/3293] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:20 2022] POST /identity/v3/projects => generated 389 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:20.859165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:20.882062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751e823-5860-4f5d-843a-6a2c56c9d3ce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:20.952671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:21.014536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.015351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.015843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.016268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.017051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.019983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.059663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1669/3294] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:20 2022] POST /identity/v3/users => generated 295 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:21.064929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.081024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97447842-041f-4471-a328-a1c51781a321 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.091148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.092586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.093423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.094670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.096054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.098312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.127405 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1626/3295] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:21.131237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.144056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f87d-5cf3-46ae-90cf-9fba7facfba7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.149996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.150795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.151179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.151556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.152541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.154393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bca157-dfc4-4a3b-99c5-ff82a66a54d2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.173304 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1670/3296] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:21.177552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.192190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e377c94-25c2-453c-9b54-ba807039c10d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.208142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.208968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.209495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.210025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] PATH_INFO: `/v3/projects/147a01a4f4f34f768accdadd818ab3e0/users/5f012723472342c69af04c14a04cf62c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.261094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147a01a4f4f34f768accdadd818ab3e0, user_id=5f012723472342c69af04c14a04cf62c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.263868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.292833 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1627/3297] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:21 2022] PUT /identity/v3/projects/147a01a4f4f34f768accdadd818ab3e0/users/5f012723472342c69af04c14a04cf62c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:21.304703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.305920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.306380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.347281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] MFA Rules not processed for user `5f012723472342c69af04c14a04cf62c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:21.349411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:21.459627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.462660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1671/3298] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:21.546599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:21.553402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116841-274f-45c2-9b48-d33649c35210 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.561762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.563691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.564513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.565021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.565911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.569261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.629810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1628/3299] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:21 2022] POST /identity/v3/projects => generated 389 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:21.639526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.657528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac62b0e-bd8e-43f2-9ab2-6284ffe42018 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.743734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.744363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.746473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.746784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.747439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.749822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.777362 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1672/3300] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:21 2022] POST /identity/v3/users => generated 301 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:21.781952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.796205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c51419-c133-44eb-8c4d-5be142eae703 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.803554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.804730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.805903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.806146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.806934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.808759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.822074 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1629/3301] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:21.825486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.836563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1833357-f383-4a84-b00d-0d4a6f033316 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:21.841948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:21.843323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:21.844289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:21.844289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] PATH_INFO: `/v3/projects/8a9586afa6644cc6b077fc8c9ac64910/users/db36d640ff6c4681a582b86932f7cbb6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:21.863722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9586afa6644cc6b077fc8c9ac64910, user_id=db36d640ff6c4681a582b86932f7cbb6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:21.866472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:21.885133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:21.891066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1673/3302] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:21 2022] PUT /identity/v3/projects/8a9586afa6644cc6b077fc8c9ac64910/users/db36d640ff6c4681a582b86932f7cbb6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:21.895877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:21.910145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e75219-5deb-412c-b830-579f3dbd9bed admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:22.037263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:22.038065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:22.038469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:22.038910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:22.039687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:22.041452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee992-e717-4933-9fb5-ebba1bd195b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:22.054937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1630/3303] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:22.063975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:22.092451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecee5b9-5b6e-42b6-bf77-53991b7dac18 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:22.109288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:22.110117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:22.110399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:22.110720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] PATH_INFO: `/v3/projects/8a9586afa6644cc6b077fc8c9ac64910/users/db36d640ff6c4681a582b86932f7cbb6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:22.113419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9586afa6644cc6b077fc8c9ac64910, user_id=db36d640ff6c4681a582b86932f7cbb6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:22.117990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:22.140460 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1674/3304] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:22 2022] PUT /identity/v3/projects/8a9586afa6644cc6b077fc8c9ac64910/users/db36d640ff6c4681a582b86932f7cbb6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:22.142666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967080e-8267-4468-92cb-f1135b5379d0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:22.143138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967080e-8267-4468-92cb-f1135b5379d0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:22.143479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967080e-8267-4468-92cb-f1135b5379d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:22.182704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9967080e-8267-4468-92cb-f1135b5379d0 None None] MFA Rules not processed for user `db36d640ff6c4681a582b86932f7cbb6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:22.184198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9967080e-8267-4468-92cb-f1135b5379d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:22.357371 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1631/3305] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:59:22.611244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:22.613854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:22.706953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce663286-b121-4948-9513-ad1def8054ad admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:22.716838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:22.717450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:22.717726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:22.717993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:22.804272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:22.805847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:22.815966 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1675/3306] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:23.478717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:23.479220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:23.479303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:23.504339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:23.505650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:23.571254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:23.593690 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1632/3307] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:23.608966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:23.610810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:23.754742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db39362-b6b6-463c-b594-9dc05e08601c service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:23.763171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:23.763907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:23.764293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:23.764689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:23.776107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:23.779413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918e419-d6dc-4eb8-8755-5b7c064a028e service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:23.801645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1676/3308] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:23.851572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:23.874288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a30197-0aad-479d-897f-45cd783faf51 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:23.881557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:23.882846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:23.883332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:23.883611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:23.893957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:23.896120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:23.908170 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1633/3309] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:24.454186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97233c6-13b0-4e22-9727-05de514da01d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:24.454861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97233c6-13b0-4e22-9727-05de514da01d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:24.455377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97233c6-13b0-4e22-9727-05de514da01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:24.491500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e97233c6-13b0-4e22-9727-05de514da01d None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:24.492691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e97233c6-13b0-4e22-9727-05de514da01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:24.499800 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1677/3310] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:24.513344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:24.530329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceba19-012c-406f-a809-b360627e12f8 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:24.536535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a9583-1b65-4058-987b-1d358124275e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:24.537312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9583-1b65-4058-987b-1d358124275e service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:24.537708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9583-1b65-4058-987b-1d358124275e service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:24.538104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a9583-1b65-4058-987b-1d358124275e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:24.547162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a9583-1b65-4058-987b-1d358124275e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:24.549058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a9583-1b65-4058-987b-1d358124275e service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:24.557988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1634/3311] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:24.972744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:24.973333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:24.973683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:24.992727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:25.001100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:25.002622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:25.087631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1678/3312] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:25.385301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:25.386621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:25.386972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:25.415701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:25.417211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:25.430618 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1635/3313] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:25.446115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:25.460529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfbbe8-bad9-41df-be4f-8564bd691a0d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:25.466603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:25.467231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:25.467508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:25.467987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:25.477401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:25.479099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba53f8c-8d71-4c72-97d8-2681bbdfed12 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:25.489206 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1679/3314] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:28.097648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:28.100094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:28.181696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516fc73-bc15-49ca-a146-f16487d8f1f0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:28.188560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:28.189454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:28.190143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:28.190712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:28.194399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:28.211395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:28.213422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:28.226371 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1636/3315] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:28.647931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936abc4-f717-4ab3-b63a-5e6e6e94fdcc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:28.648406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936abc4-f717-4ab3-b63a-5e6e6e94fdcc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:28.648483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936abc4-f717-4ab3-b63a-5e6e6e94fdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:28.682761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6936abc4-f717-4ab3-b63a-5e6e6e94fdcc None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:28.684078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6936abc4-f717-4ab3-b63a-5e6e6e94fdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:28.693676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1680/3316] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:30.323975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:30.478707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208acead-27f8-47b0-bca8-86ed89270b08 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:30.488751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:30.490260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:30.491819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:30.492254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] PATH_INFO: `/v3/users/5f012723472342c69af04c14a04cf62c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:30.497690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f012723472342c69af04c14a04cf62c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:30.502819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:30.622571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f012723472342c69af04c14a04cf62c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:30.623232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f012723472342c69af04c14a04cf62c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:30.641093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f012723472342c69af04c14a04cf62c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:30.644852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:30.662848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f012723472342c69af04c14a04cf62c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:30.663678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f012723472342c69af04c14a04cf62c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:30.684457 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1637/3317] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:30 2022] DELETE /identity/v3/users/5f012723472342c69af04c14a04cf62c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:30.689674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:30.690140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:30.690140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:30.726037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:30.727356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:30.787589 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1681/3318] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:30.797244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:30.798813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:30.857702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9286aa0-9667-4ab4-9935-f7587e8be84c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:30.862023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:30.862679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:30.862978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:30.863246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:30.872102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:30.873525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:30.882994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1638/3319] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:31.092867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:31.094873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:31.114445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a554600-3986-4ac8-8fd3-f856a5739687 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:31.126700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:31.129031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:31.129401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:31.129586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] PATH_INFO: `/v3/projects/147a01a4f4f34f768accdadd818ab3e0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:31.132861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147a01a4f4f34f768accdadd818ab3e0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:31.135911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:31.217046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147a01a4f4f34f768accdadd818ab3e0', 'request_id': 'req-31304b1c-9eca-456b-96ce-a0ab609fdb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.233793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147a01a4f4f34f768accdadd818ab3e0', 'request_id': 'req-31304b1c-9eca-456b-96ce-a0ab609fdb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.241284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147a01a4f4f34f768accdadd818ab3e0', 'request_id': 'req-31304b1c-9eca-456b-96ce-a0ab609fdb87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.260589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] The token cache is being invalidate because project 147a01a4f4f34f768accdadd818ab3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:31.260886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.261897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.263805 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1682/3320] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:31 2022] DELETE /identity/v3/projects/147a01a4f4f34f768accdadd818ab3e0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:31.268594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:31.279489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73970186-380c-4ff8-a7e5-6a68156787cc service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:31.382480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:31.383471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:31.383922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:31.384885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] PATH_INFO: `/v3/users/db36d640ff6c4681a582b86932f7cbb6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:31.391933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db36d640ff6c4681a582b86932f7cbb6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:31.398050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:31.480322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db36d640ff6c4681a582b86932f7cbb6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.481008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db36d640ff6c4681a582b86932f7cbb6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.495020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db36d640ff6c4681a582b86932f7cbb6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.505509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db36d640ff6c4681a582b86932f7cbb6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.507221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71115fdc-78bb-4821-98ca-258c44a41301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db36d640ff6c4681a582b86932f7cbb6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:31.521629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:31.538739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1639/3321] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:31 2022] DELETE /identity/v3/users/db36d640ff6c4681a582b86932f7cbb6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:32.054199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.056033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:32.082613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304b1c-9eca-456b-96ce-a0ab609fdb87 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.104368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:32.177047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.177911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.178285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.178693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] PATH_INFO: `/v3/projects/8a9586afa6644cc6b077fc8c9ac64910` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.180937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9586afa6644cc6b077fc8c9ac64910)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.183824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.272184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9586afa6644cc6b077fc8c9ac64910', 'request_id': 'req-f88ffa8e-7520-482b-a50c-e56c4051f8d7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:32.281622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9586afa6644cc6b077fc8c9ac64910', 'request_id': 'req-f88ffa8e-7520-482b-a50c-e56c4051f8d7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:32.290402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9586afa6644cc6b077fc8c9ac64910', 'request_id': 'req-f88ffa8e-7520-482b-a50c-e56c4051f8d7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:32.309067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] The token cache is being invalidate because project 8a9586afa6644cc6b077fc8c9ac64910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:32.310016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:32.310833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f88ffa8e-7520-482b-a50c-e56c4051f8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:32.313957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1683/3322] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:32 2022] DELETE /identity/v3/projects/8a9586afa6644cc6b077fc8c9ac64910 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:32.320823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.320823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.320823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.341356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:32.342830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:32.424006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1640/3323] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:32.426307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.426876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.427358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.451162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:32.452408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:32.461913 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1684/3324] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:32.466196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.478341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b139c2-d628-4c6b-81e6-575955bd9395 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.483396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.484121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.485314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.485734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.486595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:59:32.487138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.488886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.501896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1641/3325] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:59:32.506121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.515800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0a2e1-1239-475f-8177-f3d0ef3ecf3a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.521689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.522300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.522610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.522887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.523974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.525530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.558491 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1685/3326] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:32 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:32.563276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.583853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdad13-d0d6-4ab5-920d-3a7b3a571c96 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.708728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.709723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.710158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.710459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.711358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.713918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.744061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1642/3327] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:32 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:59:32.745353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.758023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889aaef4-085b-4b4d-9bfb-84cfff288e29 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.770312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.775531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.776944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.777219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.779723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.783637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.826203 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1686/3328] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:59:32.827248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.847401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52517d2c-86a4-487b-a9a6-b8abc6ebf76d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.861175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.862132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.862685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.863196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.865926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.868166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a59858-ca37-4f72-bab3-ad8641892163 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.893886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1643/3329] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:32.894587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:32.910653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a1f72-7c97-4902-ae58-6d80f9b2c996 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:32.919151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:32.920904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.921314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.922345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] PATH_INFO: `/v3/projects/82b1ace927fb4587a79ac147f5aa7df6/users/3566666c11ed499da3b3d90c080a883c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:32.943027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b1ace927fb4587a79ac147f5aa7df6, user_id=3566666c11ed499da3b3d90c080a883c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:32.946620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:32.980410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1687/3330] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:32 2022] PUT /identity/v3/projects/82b1ace927fb4587a79ac147f5aa7df6/users/3566666c11ed499da3b3d90c080a883c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:59:32.982413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:32.983108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:32.983481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.048221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] MFA Rules not processed for user `3566666c11ed499da3b3d90c080a883c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:33.049923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:33.154791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:33.190983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1644/3331] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:33.195319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.254712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:33.286003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699fcb-b1e3-46dc-a5bd-a0a9219f35ba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.298355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.299293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.299915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.300472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.301455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.304087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.345957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1688/3332] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:33 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:33.350080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.361865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbb6e4-8732-4cae-bd35-d19aede11c3d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.441414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.442058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.442219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.442524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.443455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.445332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.472478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1645/3333] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:33 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:59:33.476475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.491055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ec64e-461b-41df-b199-6e18748f007f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.497835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.498533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.498928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.499186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.499949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.501791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.514655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1689/3334] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:33.518613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.536110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27e600-b4ab-477e-8ae5-ecc595ea4106 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.541406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.542063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.542562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.542841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] PATH_INFO: `/v3/projects/a11959b29e344e69b0b65148a1c1e4e0/users/b3f801dd1e6c4725bd32515d965ef785/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.560092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11959b29e344e69b0b65148a1c1e4e0, user_id=b3f801dd1e6c4725bd32515d965ef785, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.563140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.578883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1646/3335] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:33 2022] PUT /identity/v3/projects/a11959b29e344e69b0b65148a1c1e4e0/users/b3f801dd1e6c4725bd32515d965ef785/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:33.583135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.594190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc318a2d-427d-4c41-b3af-83bcae5d6595 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.669034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.669640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.670058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.670334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.671148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.673678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369c6f3-680a-43b0-8011-fe104293f529 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.684573 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1690/3336] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:33.689142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:33.700983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ccbcb-edb3-4e32-884f-1325fd3a4575 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:33.711032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:33.712651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.712651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.712651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] PATH_INFO: `/v3/projects/a11959b29e344e69b0b65148a1c1e4e0/users/b3f801dd1e6c4725bd32515d965ef785/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.715127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11959b29e344e69b0b65148a1c1e4e0, user_id=b3f801dd1e6c4725bd32515d965ef785, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:33.718250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fc610-5219-4edb-8666-b4b0fed9e1c4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:33.729895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1647/3337] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:33 2022] PUT /identity/v3/projects/a11959b29e344e69b0b65148a1c1e4e0/users/b3f801dd1e6c4725bd32515d965ef785/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:33.741542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.741833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.742107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.790830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] MFA Rules not processed for user `b3f801dd1e6c4725bd32515d965ef785`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:33.792055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:33.892265 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1691/3338] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:33.905621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036df6-10e9-4254-91f8-99d902ac678d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:33.907167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036df6-10e9-4254-91f8-99d902ac678d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:33.907692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036df6-10e9-4254-91f8-99d902ac678d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:33.931070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e9036df6-10e9-4254-91f8-99d902ac678d None None] MFA Rules not processed for user `3566666c11ed499da3b3d90c080a883c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:33.933172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e9036df6-10e9-4254-91f8-99d902ac678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:33.992018 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1648/3339] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:34.004107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:34.005701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:34.063812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:34.113050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12034553-a13e-4652-87ae-1a0b285858bc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:34.119381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:34.120324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:34.120651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:34.121023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:34.132028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:34.133751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:34.151076 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1692/3340] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:38.591464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb697-5438-4fbc-ad8a-65d1072d32aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:38.592327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb697-5438-4fbc-ad8a-65d1072d32aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:38.592785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefb697-5438-4fbc-ad8a-65d1072d32aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:38.620295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1fefb697-5438-4fbc-ad8a-65d1072d32aa None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:38.621589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fefb697-5438-4fbc-ad8a-65d1072d32aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:38.689466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:38.705890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1649/3341] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:38.720605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:38.722044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:38.796400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d40240-d6ef-4095-ae78-b22bebd36c60 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:38.801732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:38.802351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:38.802788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:38.803065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:38.813016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:38.815040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:38.825168 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1693/3342] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 20:59:39.239930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200487-13c1-41e2-9275-c83fbd3b177e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:39.240670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200487-13c1-41e2-9275-c83fbd3b177e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:39.241367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba200487-13c1-41e2-9275-c83fbd3b177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:39.270232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ba200487-13c1-41e2-9275-c83fbd3b177e None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:39.271789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba200487-13c1-41e2-9275-c83fbd3b177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:39.281935 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1650/3343] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:39.294092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:39.306162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1b2ce-17f8-4459-b9cf-7fd7f4302e6e service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:39.311636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:39.312358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:39.312727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:39.313005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:39.321752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:39.323434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:39.334340 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1694/3344] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:39.583885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:39.584167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:39.584608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:39.610112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:39.612092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:39.621304 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1651/3345] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 20:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:39.650232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:39.677210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d64349-3136-43d5-81f1-a823304eca4b service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:39.685564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:39.686182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:39.686670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:39.686952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:39.704193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:39.705790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbe085-2fd3-4c04-8167-50a5d55a26d5 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:39.715019 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1695/3346] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 20:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 20:59:39.847652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:39.849155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:39.944495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7609e3-9dae-409e-b720-b76c9bba1d6a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:39.954324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:39.955618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:39.956047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:39.956666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] PATH_INFO: `/v3/users/00585dbe7f814eeebda43417ca1d0ffa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:39.958133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00585dbe7f814eeebda43417ca1d0ffa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:39.961259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:40.037888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00585dbe7f814eeebda43417ca1d0ffa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.039277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00585dbe7f814eeebda43417ca1d0ffa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.058275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00585dbe7f814eeebda43417ca1d0ffa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.071861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00585dbe7f814eeebda43417ca1d0ffa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.072718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00585dbe7f814eeebda43417ca1d0ffa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.097215 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1652/3347] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:39 2022] DELETE /identity/v3/users/00585dbe7f814eeebda43417ca1d0ffa => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:40.101534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.102040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.102121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.136117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:40.138127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:40.225528 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1696/3348] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:40.235416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:40.236964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:40.353636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d66ed9-8732-4fc5-bf8d-2f6f05ecfa9f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:40.357527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:40.358284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.358718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.359231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.371430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:40.391208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740675aa-fb99-4e0f-8813-bfacdb85c676 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:40.399475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:40.423352 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1653/3349] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:40.626777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:40.628272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:40.651808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92364bd-caf2-4110-b8c7-557498ebee40 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:40.659501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:40.660694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.660976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.661326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] PATH_INFO: `/v3/projects/265d7d88bb6f45708203ea74df8f3214` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.663124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265d7d88bb6f45708203ea74df8f3214)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:40.665835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:40.693791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:40.734891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265d7d88bb6f45708203ea74df8f3214', 'request_id': 'req-caab9b73-c1ab-4b91-b06f-c2266557cfe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.750636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265d7d88bb6f45708203ea74df8f3214', 'request_id': 'req-caab9b73-c1ab-4b91-b06f-c2266557cfe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.768468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265d7d88bb6f45708203ea74df8f3214', 'request_id': 'req-caab9b73-c1ab-4b91-b06f-c2266557cfe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.794130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] The token cache is being invalidate because project 265d7d88bb6f45708203ea74df8f3214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:40.794796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.796255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-caab9b73-c1ab-4b91-b06f-c2266557cfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:40.802327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1697/3350] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:40 2022] DELETE /identity/v3/projects/265d7d88bb6f45708203ea74df8f3214 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:40.807242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.808171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.808993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.836087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:40.837260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:40.909408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1654/3351] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:59:40.917446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.917739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.918021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.940897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:40.942121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:40.955770 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1698/3352] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:40.957198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:40.970696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236eff4c-66a1-4b8e-83ed-e885df224356 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:40.983220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:40.983851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:40.984195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:40.984477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:40.985135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:59:40.985437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:40.987201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:40.999439 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1655/3353] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:59:41.004471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.013991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0c42-9c65-438c-b7a7-cd4293bb60af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.019468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.020069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.020421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.020704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.021347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.023294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.054148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1699/3354] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:41 2022] POST /identity/v3/projects => generated 395 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:59:41.058072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.070754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197238-f362-45c3-85da-8523010fca02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.141596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.142280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.142584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.142862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.143504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.145428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.177583 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1656/3355] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:41 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 20:59:41.178568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.194637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a234b9-a45a-4d5f-adec-7b15942d8c8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.201436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.202269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.203267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.203545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.204644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.206304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.217679 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1700/3356] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:41.222313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.232795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340483b-9a89-40d3-b616-15813f63731b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.238751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.239358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.239768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.240053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.240787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.242408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ac109-e982-4c6b-9a72-8b33455b0eb1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.254996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1657/3357] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:41.259605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.273645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6b076-8542-4e99-85ea-2311a41a0e8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.278688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.279581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.279888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.280136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] PATH_INFO: `/v3/projects/fa12532d0ad845f5b227818ca081126d/users/eb081bddbf954c99b9e70bbb21f81fa3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.298685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa12532d0ad845f5b227818ca081126d, user_id=eb081bddbf954c99b9e70bbb21f81fa3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.301481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.319557 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1701/3358] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:41 2022] PUT /identity/v3/projects/fa12532d0ad845f5b227818ca081126d/users/eb081bddbf954c99b9e70bbb21f81fa3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:41.332092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff6a76-5b02-4071-8340-9958359f3ed1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.332677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff6a76-5b02-4071-8340-9958359f3ed1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.333528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff6a76-5b02-4071-8340-9958359f3ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.384424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1ff6a76-5b02-4071-8340-9958359f3ed1 None None] MFA Rules not processed for user `eb081bddbf954c99b9e70bbb21f81fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:41.385346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1ff6a76-5b02-4071-8340-9958359f3ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:41.471091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:41.490152 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1658/3359] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:41.527863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:41.529580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:41.619930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f2c9e-f9a8-4d20-a744-e4b32ecafd42 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:41.625815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:41.626583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.627191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.627191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.654873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:41.657312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:41.667312 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1702/3360] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:41 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:41.727502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848850-76c6-4885-ad38-8cec6c6811e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:41.727703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848850-76c6-4885-ad38-8cec6c6811e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:41.727979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848850-76c6-4885-ad38-8cec6c6811e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:41.747868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e0848850-76c6-4885-ad38-8cec6c6811e5 None None] MFA Rules not processed for user `eb081bddbf954c99b9e70bbb21f81fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:41.749086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e0848850-76c6-4885-ad38-8cec6c6811e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:41.756563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1659/3361] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:42.189306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:42.242612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:42.283023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a26aa-e0cc-43f8-8512-2a9222c91139 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:42.287682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:42.288333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:42.288611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:42.289080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] PATH_INFO: `/v3/users/eb081bddbf954c99b9e70bbb21f81fa3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:42.290218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb081bddbf954c99b9e70bbb21f81fa3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:42.292177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:42.341587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb081bddbf954c99b9e70bbb21f81fa3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:42.342244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb081bddbf954c99b9e70bbb21f81fa3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:42.350713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb081bddbf954c99b9e70bbb21f81fa3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:42.365399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb081bddbf954c99b9e70bbb21f81fa3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:42.366173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb081bddbf954c99b9e70bbb21f81fa3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:42.385472 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1703/3362] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:42 2022] DELETE /identity/v3/users/eb081bddbf954c99b9e70bbb21f81fa3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:42.387508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:42.388296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:42.388575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:42.410304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:42.411578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:42.512836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1660/3363] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:42.534611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:42.536871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:42.616235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0286e-a4f2-4497-a7fd-6fb22b94c5da admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:42.630278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:42.631260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:42.631398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:42.631641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:42.644508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:42.646048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8dc582-8054-4da7-a631-740c36d603b0 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:42.660867 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1704/3364] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:43.229250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.230838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:43.248881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2bfe1-933b-4003-9e49-0fbd4a7aac73 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.255151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.255801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.256075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.256353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] PATH_INFO: `/v3/projects/fa12532d0ad845f5b227818ca081126d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.257509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa12532d0ad845f5b227818ca081126d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.259486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.330332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa12532d0ad845f5b227818ca081126d', 'request_id': 'req-cd341244-aabd-49a8-825e-19df4afd033f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:43.338274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa12532d0ad845f5b227818ca081126d', 'request_id': 'req-cd341244-aabd-49a8-825e-19df4afd033f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:43.353476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa12532d0ad845f5b227818ca081126d', 'request_id': 'req-cd341244-aabd-49a8-825e-19df4afd033f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:43.381952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] The token cache is being invalidate because project fa12532d0ad845f5b227818ca081126d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:43.382289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:43.383279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd341244-aabd-49a8-825e-19df4afd033f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:43.385106 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1661/3365] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:43 2022] DELETE /identity/v3/projects/fa12532d0ad845f5b227818ca081126d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:43.389666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.390168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.390270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.414667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:43.416373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:43.490859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:43.505398 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1705/3366] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 20:59:43.513260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.513768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.513871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.533885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:43.535400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:43.543461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1662/3367] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:43.548311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.558107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a9a53-b8aa-431d-9556-01e8da5dff33 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.563547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.564156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.564439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.564703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.565360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:59:43.565667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.567388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.580192 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1706/3368] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 20:59:43.584433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.594705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbb4c7-9aae-409d-8cbb-5a77e08ae14f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.599403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.600184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.600655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.600914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.601720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.603838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.654960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1663/3369] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:43 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:43.659010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.677199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320b134-dee4-40c6-8ea6-8fc26053f0a5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.782404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.783286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.783745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.784722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.791056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.798528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.841997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1707/3370] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:43 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:43.846698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.858089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07c771-af3d-4067-8dd2-6025b52fc81a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.866952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.867782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.868374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.869086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.869882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.883759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.903330 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1664/3371] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:43.907631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.918453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0325f-eacb-4883-a23b-58c26d4a19bb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.925055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.925055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.925055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.925389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:43.936631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:43.938582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf22f3-15ea-4391-adf8-f4ca602cac40 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:43.977218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:43.977372 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1708/3372] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:43.990718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af827683-bb6e-4b8a-b84a-768aa21b8843 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:43.995641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:43.996306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:43.996618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:43.996888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] PATH_INFO: `/v3/projects/981fdf8ae7524612b9c2f6a3f16c39ec/users/5ddab9c74c004e74a51fd8008886e542/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:44.012977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981fdf8ae7524612b9c2f6a3f16c39ec, user_id=5ddab9c74c004e74a51fd8008886e542, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:44.016002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:44.029856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1665/3373] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:43 2022] PUT /identity/v3/projects/981fdf8ae7524612b9c2f6a3f16c39ec/users/5ddab9c74c004e74a51fd8008886e542/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 20:59:44.036203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:44.036421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:44.036693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:44.078988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:44.081146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:44.178842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1709/3374] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:44.189032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:44.191365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:44.207395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:44.288583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2c1e9-4d97-49d7-9ffa-fb39bca390d3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:44.311745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:44.315724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:44.316208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:44.316602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:44.336139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:44.338367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:44.352768 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1666/3375] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:47.942449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:47.943959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:48.132465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:48.147363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5987ad-d4c7-4a2a-917a-2b1b66dffe94 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:48.148957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:48.150601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:48.166741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:48.167762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:48.168047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:48.168321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:48.174629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:48.189409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559ec2d-5061-4223-b377-98cc41bc8d89 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:48.208145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:48.208956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:48.209341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:48.210439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] PATH_INFO: `/v3/users/6a38dd114ec04ba5826a68a9e796ab8d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:48.213394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a38dd114ec04ba5826a68a9e796ab8d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:48.215768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:48.216330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:48.220103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:48.249334 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1710/3376] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:48.339971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a38dd114ec04ba5826a68a9e796ab8d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:48.340657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a38dd114ec04ba5826a68a9e796ab8d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:48.353193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a38dd114ec04ba5826a68a9e796ab8d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:48.368395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a38dd114ec04ba5826a68a9e796ab8d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:48.370338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a38dd114ec04ba5826a68a9e796ab8d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:48.456616 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1667/3377] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 20:59:48 2022] DELETE /identity/v3/users/6a38dd114ec04ba5826a68a9e796ab8d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:48.910951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:48.927567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088569eb-91e0-4c3b-889b-b89ced8a5bc0 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:48.939886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:48.945722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:49.039238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5b98f-b4eb-42e9-bf93-917f6709f253 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.044097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.044753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.045036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.045412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.048406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.048884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:49.049715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.050250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.050763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] PATH_INFO: `/v3/projects/ec085cc2148f4cda9483fbe7257eeb14` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.052101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec085cc2148f4cda9483fbe7257eeb14)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.054667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.109581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:49.132043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec085cc2148f4cda9483fbe7257eeb14', 'request_id': 'req-44331e3e-4013-4667-a675-b67c08f80472'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:49.139056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec085cc2148f4cda9483fbe7257eeb14', 'request_id': 'req-44331e3e-4013-4667-a675-b67c08f80472'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:49.146800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec085cc2148f4cda9483fbe7257eeb14', 'request_id': 'req-44331e3e-4013-4667-a675-b67c08f80472'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:49.165075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] The token cache is being invalidate because project ec085cc2148f4cda9483fbe7257eeb14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 20:59:49.165474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:49.166297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44331e3e-4013-4667-a675-b67c08f80472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 20:59:49.168161 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1711/3378] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 20:59:48 2022] DELETE /identity/v3/projects/ec085cc2148f4cda9483fbe7257eeb14 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:49.168710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.170868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab40ac-ef40-4659-8ec4-8db2fabada00 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.174009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.174310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.174624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.188003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1668/3379] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:49.232461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:49.239361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:49.355267 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1712/3380] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:49.364902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.365335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.365706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.395523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:49.397199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:49.414669 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1669/3381] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:49.415267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.433136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a8832-dd8e-4e4f-8351-ce736d0e86f8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.439071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.439777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.440144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.440484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.441207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 20:59:49.441581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.443416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.458568 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1713/3382] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 20:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 20:59:49.462771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.474842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e47ad-6f35-4bb2-a218-ad3f4646b9ee None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.482428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.482811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.483092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.483365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.484780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.488894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.532324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1670/3383] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 20:59:49 2022] POST /identity/v3/projects => generated 399 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:49.537096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.554024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ae850-04ce-4222-bd72-abb4a5a0fb13 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.559511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:49.687687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.689346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.690788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.692764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.694050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.696895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.745106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1714/3384] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 20:59:49 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 20:59:49.749484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.769400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc2e21-9aa0-4bed-9c3a-8b8e6e672fd8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.785144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.785778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.786056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.787900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.789356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.793756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.825300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1671/3385] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 20:59:49.826456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.846586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086959-0d4e-4c26-aba6-3daa68474d5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.864703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.865364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.866053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.866160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.866833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:49.872526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bbd96-dde6-4bac-bd87-35337be9e3c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:49.910065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:49.926687 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1715/3386] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 20:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 20:59:49.928097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d336e79-3d11-4d17-8b86-b516bd78b6fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:49.944605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:49.947489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:49.948557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:49.949461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] PATH_INFO: `/v3/projects/18a512ecd53242fb8dd25a0367ae8180/users/81a689c647524dfdb492184b90c53b05/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:49.997805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a512ecd53242fb8dd25a0367ae8180, user_id=81a689c647524dfdb492184b90c53b05, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:50.012463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:50.059776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1672/3387] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 20:59:49 2022] PUT /identity/v3/projects/18a512ecd53242fb8dd25a0367ae8180/users/81a689c647524dfdb492184b90c53b05/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 20:59:50.066021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03e2a-e6c4-4867-9df1-193c90691556 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:50.066021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03e2a-e6c4-4867-9df1-193c90691556 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:50.066625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab03e2a-e6c4-4867-9df1-193c90691556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:50.101141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1ab03e2a-e6c4-4867-9df1-193c90691556 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:50.102399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ab03e2a-e6c4-4867-9df1-193c90691556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:50.295759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1716/3388] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:50.348701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:50.351619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:50.505002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4696697-7d72-4c79-8317-758e7f1f6a0b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:50.509064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:50.509961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:50.510953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:50.511380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:50.525860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:50.527607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:50.543685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1673/3389] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:52.008088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd15-9efd-471b-9f5e-2287594d9f09 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:52.008527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd15-9efd-471b-9f5e-2287594d9f09 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:52.008631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd15-9efd-471b-9f5e-2287594d9f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:52.050691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f2eadd15-9efd-471b-9f5e-2287594d9f09 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:52.051946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2eadd15-9efd-471b-9f5e-2287594d9f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:52.141110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1717/3390] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:52.738877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:52.750349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27a491-c586-4e54-a87e-57bd8304c1d0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:52.773212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:52.774239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:52.774762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:52.775341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:52.780101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:52.920055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:52.949102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:52.954846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:52.975763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1674/3391] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:54.688172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd80f6-1275-4568-9799-64a651cf90d1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:54.688578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd80f6-1275-4568-9799-64a651cf90d1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:54.688687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd80f6-1275-4568-9799-64a651cf90d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:54.786459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2fcd80f6-1275-4568-9799-64a651cf90d1 None None] MFA Rules not processed for user `5ddab9c74c004e74a51fd8008886e542`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:54.788209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2fcd80f6-1275-4568-9799-64a651cf90d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:54.822343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 20:59:54.917052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1718/3392] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 20:59:54.929050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 20:59:54.931248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:54.957534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffcdfc-6dff-41b8-b4b3-b7332956a4a3 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 20:59:54.963752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 20:59:54.964779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:54.966114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:54.970843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:54.988406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 20:59:54.991132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22c16c-380f-4042-9d39-8185d4dfa0f7 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 20:59:55.005645 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1675/3393] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 20:59:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 20:59:56.265929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23c72-d45b-4f94-85d3-1338f1f8d575 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 20:59:56.266748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23c72-d45b-4f94-85d3-1338f1f8d575 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 20:59:56.266748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23c72-d45b-4f94-85d3-1338f1f8d575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 20:59:56.306749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-70f23c72-d45b-4f94-85d3-1338f1f8d575 None None] MFA Rules not processed for user `5ddab9c74c004e74a51fd8008886e542`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 20:59:56.308346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-70f23c72-d45b-4f94-85d3-1338f1f8d575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 20:59:56.318534 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1719/3394] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 20:59:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:01.255822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:01.260808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:01.261420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:01.313386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] MFA Rules not processed for user `81a689c647524dfdb492184b90c53b05`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:01.316057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:01.479080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1676/3395] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:01.489499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:01.489985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:01.489985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:01.521825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] MFA Rules not processed for user `81a689c647524dfdb492184b90c53b05`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:01.523134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:01.534868 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1720/3396] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:01.548428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:01.565307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660d21c-8c4a-4b25-8fd5-77cf0fbd6b31 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:01.572385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:01.573647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:01.574328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:01.574865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:01.585646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:01.588135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:01.606989 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1677/3397] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:03.195856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:03.198202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:03.357761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce05a55-ef33-448d-b2a8-0d7ca79f4bc7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:03.366376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:03.367832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:03.368284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:03.368712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:03.373195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:03.390569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:03.392450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:03.411504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1721/3398] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:07.668872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:07.671012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:07.701735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a54c2-eac7-4173-8082-d18e740f84aa service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:07.708698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:07.709748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:07.710122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:07.710562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] PATH_INFO: `/v3/users/31761ade4b4a4df580fd5de55426eb0b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:07.715610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31761ade4b4a4df580fd5de55426eb0b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:07.719705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:07.800378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31761ade4b4a4df580fd5de55426eb0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:07.801029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31761ade4b4a4df580fd5de55426eb0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:07.813935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31761ade4b4a4df580fd5de55426eb0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:07.823258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31761ade4b4a4df580fd5de55426eb0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:07.824393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31761ade4b4a4df580fd5de55426eb0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:07.851532 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1678/3399] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:00:07 2022] DELETE /identity/v3/users/31761ade4b4a4df580fd5de55426eb0b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:08.088664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:08.109348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806108a0-5fc3-44a3-b335-6272a3471df2 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:08.198399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:08.199534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:08.200150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:08.200778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] PATH_INFO: `/v3/projects/c4004a1dbc2441f8a5c4ace42af3e2df` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:08.204996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4004a1dbc2441f8a5c4ace42af3e2df)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:08.212618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:08.292027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:08.307083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4004a1dbc2441f8a5c4ace42af3e2df', 'request_id': 'req-c60d7c1a-33bc-4b5a-a039-89de55f37876'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.315036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4004a1dbc2441f8a5c4ace42af3e2df', 'request_id': 'req-c60d7c1a-33bc-4b5a-a039-89de55f37876'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.322872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4004a1dbc2441f8a5c4ace42af3e2df', 'request_id': 'req-c60d7c1a-33bc-4b5a-a039-89de55f37876'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.362884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] The token cache is being invalidate because project c4004a1dbc2441f8a5c4ace42af3e2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:00:08.363547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.364950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.368486 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1722/3400] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:00:08 2022] DELETE /identity/v3/projects/c4004a1dbc2441f8a5c4ace42af3e2df => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:08.373244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:08.375375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:08.417973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978d4cd-141b-4838-a30b-cdfceec7546f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:08.486562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:08.568910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:08.569799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:08.570112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:08.570387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] PATH_INFO: `/v3/users/e963d81423304dadbbf445c18661e767` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:08.576415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e963d81423304dadbbf445c18661e767)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:08.578436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:08.696663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e963d81423304dadbbf445c18661e767'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.698136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e963d81423304dadbbf445c18661e767'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.716955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e963d81423304dadbbf445c18661e767'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.736193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e963d81423304dadbbf445c18661e767'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.737155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c39b3a1-330f-4f92-b635-8cf4d1c7234d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e963d81423304dadbbf445c18661e767'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:08.777278 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1679/3401] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:00:08 2022] DELETE /identity/v3/users/e963d81423304dadbbf445c18661e767 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:09.162135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.164051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:09.198023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7c1a-33bc-4b5a-a039-89de55f37876 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.261720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.262368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.264436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.264922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] PATH_INFO: `/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.266303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f3f539c41f4979a3f8f7f268f5357b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.268268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.323528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f3f539c41f4979a3f8f7f268f5357b', 'request_id': 'req-8c497057-f0f0-4937-84cc-d113590e9a97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:09.330265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f3f539c41f4979a3f8f7f268f5357b', 'request_id': 'req-8c497057-f0f0-4937-84cc-d113590e9a97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:09.336768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f3f539c41f4979a3f8f7f268f5357b', 'request_id': 'req-8c497057-f0f0-4937-84cc-d113590e9a97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:09.350709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] The token cache is being invalidate because project a4f3f539c41f4979a3f8f7f268f5357b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:00:09.351021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:09.352148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c497057-f0f0-4937-84cc-d113590e9a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:09.353806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1723/3402] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:00:09 2022] DELETE /identity/v3/projects/a4f3f539c41f4979a3f8f7f268f5357b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:09.360194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.360699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.360953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.401266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:09.402817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:09.514780 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1680/3403] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:09.522132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.522723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.523084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.538582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.540218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:09.552620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:09.554596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:09.570608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1724/3404] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:09.573046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.592384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6586a-05dd-4b08-8f3c-3d8deb786f65 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.602821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:09.610058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.611529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.612593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.613356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.614172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:00:09.614476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.616553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.628722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545188ec-7a21-4f45-af17-318d45af9204 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.632588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.633340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.633625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.634042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.640417 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1725/3405] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:00:09.642643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.651688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:09.666869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.670277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.682595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abef104-78d2-4cb2-bc42-6c8e774503ec admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.690280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.691733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.692086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.692665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.694389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1681/3406] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:00:09.694679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.696903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.752660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1726/3407] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:00:09 2022] POST /identity/v3/projects => generated 395 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:00:09.753725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.782466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e3e0-cf05-4c49-849a-6c45e774d632 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.861186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.861794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.862234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.862701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.863522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.865605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.901812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1682/3408] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:00:09 2022] POST /identity/v3/users => generated 298 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:00:09.906331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.920312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512cd0a-8d12-4c38-b448-3b28cc0a1355 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.927159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.928047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.928475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.928952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.929755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.932160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.946514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1727/3409] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:00:09.951713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.963813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991c26-8cea-49dd-afcd-4eab8ee21ba6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:09.969905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:09.970827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:09.971101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:09.971518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:09.972325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:09.974088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec41ba-b318-446e-9ac2-6831f7e98fdd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:09.987522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1683/3410] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:00:09.992280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:09.999770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:10.012918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64d62-07dc-4ca2-8dd6-176073b9177a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:10.019120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:10.019789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:10.020062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:10.020319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] PATH_INFO: `/v3/projects/d4655b536ca04896993223f34d172298/users/f41f618458eb48f6bb4631fd254addc0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:10.057772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4655b536ca04896993223f34d172298, user_id=f41f618458eb48f6bb4631fd254addc0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:10.060525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:10.082174 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1728/3411] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:00:09 2022] PUT /identity/v3/projects/d4655b536ca04896993223f34d172298/users/f41f618458eb48f6bb4631fd254addc0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:10.087191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:10.087479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:10.087753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:10.117482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:10.118658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:10.275080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1684/3412] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:10.291474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:10.293176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:10.438549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689169e-c863-4fc3-aa4c-0aacaa446a76 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:10.443701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:10.444365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:10.444644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:10.444913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:10.457167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:10.459524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:10.471585 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1729/3413] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:11.575243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:11.589341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf79c8-b85f-40d0-8a00-019ae20ec843 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:11.599760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:11.600471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:11.600824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:11.601170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:11.603223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:11.615752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:11.617278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac11be-1e00-4ed7-9f81-5672578b40f1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:11.634967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1685/3414] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:16.633356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:16.635558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:16.737169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9a722-1417-4ea1-95db-504257fe33a5 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:16.746031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:16.747245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:16.747532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:16.747806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:16.750386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:16.765773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:16.767420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86821b74-e67a-4ccb-b2d9-88bc2f13fafd service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:16.790714 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1730/3415] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:17.919248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:17.919746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:17.919859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:17.950840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] MFA Rules not processed for user `f41f618458eb48f6bb4631fd254addc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:17.952795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:18.008590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:18.057271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1686/3416] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:18.068222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665db73d-144b-4262-85e9-e4b34df9910e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:18.068796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665db73d-144b-4262-85e9-e4b34df9910e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:18.068796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665db73d-144b-4262-85e9-e4b34df9910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:18.090939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-665db73d-144b-4262-85e9-e4b34df9910e None None] MFA Rules not processed for user `f41f618458eb48f6bb4631fd254addc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:18.093251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-665db73d-144b-4262-85e9-e4b34df9910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:18.119544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1731/3417] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:18.130567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:18.146725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081c2d2-ccb9-424e-8cf1-39aaed01a8a9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:18.152084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:18.152791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:18.153162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:18.153557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:18.163255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:18.165632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:18.176330 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1687/3418] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:18 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:22.299974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:22.300877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:22.301323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:22.349601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] MFA Rules not processed for user `c84e7493d0ed47aca659dc925e8ca39c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:22.351210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:22.463772 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1732/3419] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:23.376350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:23.395909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b818c13-d327-4995-8014-c57cea076815 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:23.408540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:23.409593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:23.409934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:23.410271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:23.412301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:23.447171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:23.448794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbb068-9604-4212-9872-d8dceb7c5134 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:23.462339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1688/3420] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:23.915895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:23.932042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156e55e-bd87-42b5-b83f-e719d0afade2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:23.939374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:23.939992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:23.940279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:23.940553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:23.943014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:23.952737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:24.036737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:24.039047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:24.051542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1733/3421] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:24.812464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef1ba-2d72-4d20-8b52-32a62e54670c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:24.813212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef1ba-2d72-4d20-8b52-32a62e54670c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:24.813634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef1ba-2d72-4d20-8b52-32a62e54670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:24.890229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-372ef1ba-2d72-4d20-8b52-32a62e54670c None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:24.892320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-372ef1ba-2d72-4d20-8b52-32a62e54670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:25.007677 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1689/3422] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:00:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:25.088997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:25.090505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:25.252445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05c19b-3e62-49b0-8977-a2164bca99ba service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:25.258357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:25.259366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:25.259878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:25.260323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:25.270596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:25.272665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:25.286376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1734/3423] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:00:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:00:26.224534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5c4c4-f773-4601-b303-13e05871a670 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:26.225114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5c4c4-f773-4601-b303-13e05871a670 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:26.225211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5c4c4-f773-4601-b303-13e05871a670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:26.256658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3bc5c4c4-f773-4601-b303-13e05871a670 None None] MFA Rules not processed for user `5ddab9c74c004e74a51fd8008886e542`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:26.258830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3bc5c4c4-f773-4601-b303-13e05871a670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:26.365960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1690/3424] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:00:28.942855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:28.945723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:28.971608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003508b-d0bc-4ac1-97bc-8d1ef95cefd5 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:28.977788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:28.979058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:28.980195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:28.980699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:28.982770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:29.001691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:29.004976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:29.022406 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1735/3425] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:34.757050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:34.757600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:34.758161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:34.784789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] MFA Rules not processed for user `f41f618458eb48f6bb4631fd254addc0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:34.786127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:34.799835 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1691/3426] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:38.910609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:38.912162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:38.936123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de770dc8-5e4e-44de-af23-e67e0452d7e4 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:38.947373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:38.948797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:38.949882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:38.960615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] PATH_INFO: `/v3/users/81a689c647524dfdb492184b90c53b05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:38.970539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a689c647524dfdb492184b90c53b05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:38.973010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:39.072806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a689c647524dfdb492184b90c53b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.074312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a689c647524dfdb492184b90c53b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.079894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:39.087491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a689c647524dfdb492184b90c53b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.110370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a689c647524dfdb492184b90c53b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.111276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7133d536-9369-444c-8cff-d27349c3b572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a689c647524dfdb492184b90c53b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.127228 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1736/3427] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:00:38 2022] DELETE /identity/v3/users/81a689c647524dfdb492184b90c53b05 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:39.399669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:39.414104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a6813-b2ec-498e-a518-ffbba14bbb2a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:39.449763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:39.499881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:39.501096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:39.501604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:39.502031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] PATH_INFO: `/v3/projects/18a512ecd53242fb8dd25a0367ae8180` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:39.503481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a512ecd53242fb8dd25a0367ae8180)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:39.505402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:39.583193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a512ecd53242fb8dd25a0367ae8180', 'request_id': 'req-54b46b80-30a9-4e42-8697-55f52a5fb52d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.605897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a512ecd53242fb8dd25a0367ae8180', 'request_id': 'req-54b46b80-30a9-4e42-8697-55f52a5fb52d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.619268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a512ecd53242fb8dd25a0367ae8180', 'request_id': 'req-54b46b80-30a9-4e42-8697-55f52a5fb52d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.646479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] The token cache is being invalidate because project 18a512ecd53242fb8dd25a0367ae8180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:00:39.646856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.647463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54b46b80-30a9-4e42-8697-55f52a5fb52d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:00:39.649319 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1692/3428] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:00:39 2022] DELETE /identity/v3/projects/18a512ecd53242fb8dd25a0367ae8180 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:39.653978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:39.654267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:39.654632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:39.689608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:39.690785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:39.808607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1737/3429] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:39.816627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:39.817404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:39.817404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:39.856322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:39.858098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:39.910558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1693/3430] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:39.910782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:39.922248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf34c0f-d929-4da5-aff9-c3105ae1746b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:39.927603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:39.929153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:39.929847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:39.931470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:39.932680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:00:39.933376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:39.935375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:39.960295 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1738/3431] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:00:39.964360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:39.979418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a48d8-0c98-404a-885a-a28371071eef None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:39.987758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:39.989154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:39.989644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:39.990102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:39.990924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.000711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.037511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1694/3432] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:00:39 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:00:40.041370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:40.053139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0c597-3935-450b-8baa-f601afff6cc1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:40.132198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:40.133124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.133581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.134007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.134934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.137730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.168139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1739/3433] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:00:40 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:00:40.172622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:40.189871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9da92-2c19-4ffa-8553-da778364288e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:40.198411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:40.199056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.199336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.199607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.200253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.209327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.229834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1695/3434] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:00:40.230879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:40.250183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122740df-df96-43e5-906b-476e17dc50f8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:40.255454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:40.256284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.256716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.257157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.257947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.260375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7ad8-35c4-49eb-a1ac-8a8f5718937c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.279737 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1740/3435] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:00:40.284078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:40.301156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1f8c-16fa-406c-94f1-ef44587bf0ac admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:40.306665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:40.307483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.307772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.308049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] PATH_INFO: `/v3/projects/5b68f292633a49ca82fb05d18ea83142/users/c7c49bed91bb437087a8ad2d865c2647/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.328994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b68f292633a49ca82fb05d18ea83142, user_id=c7c49bed91bb437087a8ad2d865c2647, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.331787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.361424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1696/3436] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:00:40 2022] PUT /identity/v3/projects/5b68f292633a49ca82fb05d18ea83142/users/c7c49bed91bb437087a8ad2d865c2647/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:00:40.363856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95a417-6ab2-4930-bbbe-730261cb1898 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.364742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95a417-6ab2-4930-bbbe-730261cb1898 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.365046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95a417-6ab2-4930-bbbe-730261cb1898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.400709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5a95a417-6ab2-4930-bbbe-730261cb1898 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:40.402975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a95a417-6ab2-4930-bbbe-730261cb1898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:40.444007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:40.491607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1741/3437] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:00:40.501693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:40.504059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:40.606062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ea16b-d15f-45ce-84d5-d2b5d0d109a8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:40.610199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:40.610853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:40.611128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:40.611396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:40.619016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:40.622557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:40.624323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:40.652597 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1697/3438] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:44.603731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d5cb-8963-4a60-b333-d766b0e9f24f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:44.604307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d5cb-8963-4a60-b333-d766b0e9f24f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:44.604307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926d5cb-8963-4a60-b333-d766b0e9f24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:44.637153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c926d5cb-8963-4a60-b333-d766b0e9f24f None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:44.638464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c926d5cb-8963-4a60-b333-d766b0e9f24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:44.709631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1742/3439] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:44.723363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:44.725505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:44.854587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdc62d-f862-4f55-8902-ed3ab3653d91 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:44.863272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:44.864211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:44.864645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:44.865068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:44.887866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:44.890058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a8bc7-a11c-45a6-a356-e36f1a008161 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:44.901346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1698/3440] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:00:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:00:47.617423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:47.617900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:47.617995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:47.690727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] MFA Rules not processed for user `c7c49bed91bb437087a8ad2d865c2647`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:47.692002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:47.885079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1743/3441] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:47.897564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8dfa6-d20f-457a-be39-a4aae5fc0e62 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:47.898451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8dfa6-d20f-457a-be39-a4aae5fc0e62 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:47.898914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8dfa6-d20f-457a-be39-a4aae5fc0e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:47.938337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-84d8dfa6-d20f-457a-be39-a4aae5fc0e62 None None] MFA Rules not processed for user `c7c49bed91bb437087a8ad2d865c2647`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:47.941512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84d8dfa6-d20f-457a-be39-a4aae5fc0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:47.954847 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1699/3442] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:00:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:48.669382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:48.672472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:48.823763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73098d14-bf34-4a52-9113-a53c94997ffd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:48.839099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:48.840118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:48.840497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:48.840871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:48.850589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:00:48.860913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:48.862574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:48.887485 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1744/3443] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:00:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:00:57.677493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:57.678118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:57.678118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:57.720835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:00:57.723291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:00:57.753035 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1700/3444] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:00:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:00:57.783163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:00:57.804911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c12568-db77-4c56-adee-c4a963d9a2a4 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:00:57.815354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:00:57.816305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:00:57.822932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:00:57.823293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:00:57.847691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:00:57.852122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82c37-08b4-410e-b37c-ab7dd14d603f service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:00:57.878686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1745/3445] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:00:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:01:07.710666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=40 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:07.866898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:13.183291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:13.184748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:13.214233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee9091-6d0e-4885-94f9-8ccbcfcfe9de None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:13.225422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:13.226082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:13.226367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:13.226798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] PATH_INFO: `/v3/users/3566666c11ed499da3b3d90c080a883c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:13.228006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3566666c11ed499da3b3d90c080a883c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:13.229981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:13.333115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:13.334047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:13.334477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:13.362622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3566666c11ed499da3b3d90c080a883c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:13.364413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3566666c11ed499da3b3d90c080a883c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:13.373908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:13.380475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:13.382595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3566666c11ed499da3b3d90c080a883c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:13.394807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3566666c11ed499da3b3d90c080a883c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:13.402849 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1746/3446] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:01:13.409737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99b1ad42-7da3-4ad1-8ba5-76abeba886f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3566666c11ed499da3b3d90c080a883c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:13.438347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:13.439894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:13.441453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1701/3447] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:01:13 2022] DELETE /identity/v3/users/3566666c11ed499da3b3d90c080a883c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:13.446836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:13.448735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:13.449208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:13.460631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4ed2-8206-4d0f-9f58-290d772236f1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:13.480764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:13.482311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:13.532813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:13.533896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:13.534416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:13.534950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:13.608483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:13.610809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:13.623429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1747/3448] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:01:13.651156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1702/3449] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:13.665337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:13.667122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:13.920567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4eae-6a33-4ed4-98ad-9fdc623b20d1 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:13.930030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:13.933331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:13.933893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:13.934424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:13.946954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:13.948884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:13.964330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1748/3450] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:14.877316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:14.879497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:14.918364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908e785-478c-404b-8e75-a002d826f2f1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:14.941131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:14.955133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:14.958629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:14.959194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] PATH_INFO: `/v3/projects/82b1ace927fb4587a79ac147f5aa7df6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:14.961033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b1ace927fb4587a79ac147f5aa7df6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:14.967703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:15.052239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:15.055112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:15.071397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63275534-9d16-446c-b427-0cfcda0e40c1 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:15.090108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b1ace927fb4587a79ac147f5aa7df6', 'request_id': 'req-12154453-6088-459d-8137-c74128eaeab6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.101284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b1ace927fb4587a79ac147f5aa7df6', 'request_id': 'req-12154453-6088-459d-8137-c74128eaeab6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.113325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b1ace927fb4587a79ac147f5aa7df6', 'request_id': 'req-12154453-6088-459d-8137-c74128eaeab6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.119264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:15.133162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] The token cache is being invalidate because project 82b1ace927fb4587a79ac147f5aa7df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:01:15.133700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.134577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.136586 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1703/3451] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:01:14 2022] DELETE /identity/v3/projects/82b1ace927fb4587a79ac147f5aa7df6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:15.140641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:15.142653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:15.174404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:15.215370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:15.216823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:15.217287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:15.217717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:15.221019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:15.237601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:15.240462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:15.259142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12154453-6088-459d-8137-c74128eaeab6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:15.263624 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1749/3452] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:01:15.265847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:15.266515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:15.266812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:15.267547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] PATH_INFO: `/v3/users/b3f801dd1e6c4725bd32515d965ef785` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:15.269531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f801dd1e6c4725bd32515d965ef785)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:15.273481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:15.366362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f801dd1e6c4725bd32515d965ef785'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.368523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f801dd1e6c4725bd32515d965ef785'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.387032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f801dd1e6c4725bd32515d965ef785'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.400393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f801dd1e6c4725bd32515d965ef785'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.404465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f801dd1e6c4725bd32515d965ef785'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:15.442931 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1704/3453] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:01:15 2022] DELETE /identity/v3/users/b3f801dd1e6c4725bd32515d965ef785 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:16.032691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:16.036579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:16.138708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c52e7-0ed4-4a02-ba6a-37758bde76c3 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:16.223516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:16.224053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:16.224328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:16.224603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:16.226564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:16.252419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:16.254076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907225d1-b74e-4a12-9da5-bee0ed7eefed service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:16.278172 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1750/3454] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:16.577711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:16.579294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:16.741632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc076d38-f274-46b1-b9f5-356e5fbddb7f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:16.752724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:16.754027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:16.754546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:16.755273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] PATH_INFO: `/v3/projects/a11959b29e344e69b0b65148a1c1e4e0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:16.766257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11959b29e344e69b0b65148a1c1e4e0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:16.771074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:16.814919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:16.907840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11959b29e344e69b0b65148a1c1e4e0', 'request_id': 'req-5b80f4c8-567f-424b-a3a4-62464bb56dbd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:16.932219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11959b29e344e69b0b65148a1c1e4e0', 'request_id': 'req-5b80f4c8-567f-424b-a3a4-62464bb56dbd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:16.943537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11959b29e344e69b0b65148a1c1e4e0', 'request_id': 'req-5b80f4c8-567f-424b-a3a4-62464bb56dbd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:16.976285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] The token cache is being invalidate because project a11959b29e344e69b0b65148a1c1e4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:01:16.976738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:16.977392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:16.982380 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1705/3455] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:01:16 2022] DELETE /identity/v3/projects/a11959b29e344e69b0b65148a1c1e4e0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:16.987434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de049ce-281d-4e96-9619-a12c03319a70 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:16.988030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de049ce-281d-4e96-9619-a12c03319a70 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:16.988462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de049ce-281d-4e96-9619-a12c03319a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.033656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1de049ce-281d-4e96-9619-a12c03319a70 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:17.041342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1de049ce-281d-4e96-9619-a12c03319a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.096175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.097793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.187246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1751/3456] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:17.188806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.189093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.189366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.207823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f4c8-567f-424b-a3a4-62464bb56dbd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.217358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.218113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.218543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.219063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.222521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.235966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:17.237313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.248211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.249764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.253109 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1752/3457] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:01:17.254302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.257516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.261771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1706/3458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:17.262636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.275153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106ec53-feb8-49f8-8322-38fd50a6790c service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.277360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed100ada-3952-4596-8b2b-563b45711edf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.280333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.281205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.281487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.281761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.282692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:01:17.283004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.284380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.284576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.285142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.285490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.285835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] PATH_INFO: `/v3/users/c84e7493d0ed47aca659dc925e8ca39c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.288440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c84e7493d0ed47aca659dc925e8ca39c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.290464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.292596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:17.308919 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1707/3459] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:01:17.310606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.323250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581f31-054a-4751-a294-6b1d37e6f780 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.329528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.330360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.330584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.330825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.331461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.333850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.356392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c84e7493d0ed47aca659dc925e8ca39c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.359148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c84e7493d0ed47aca659dc925e8ca39c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.368184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c84e7493d0ed47aca659dc925e8ca39c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.368547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1708/3460] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:01:17.374275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.376749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.390728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c84e7493d0ed47aca659dc925e8ca39c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.391578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c84e7493d0ed47aca659dc925e8ca39c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.411367 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1753/3461] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:01:17 2022] DELETE /identity/v3/users/c84e7493d0ed47aca659dc925e8ca39c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:17.458486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f1953-27f5-4efb-8b07-6b8b45f2068a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.469075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:17.533483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.534335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.534672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.534951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.535587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.537530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.567243 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1709/3462] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:01:17.568728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.579582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430fac9-4dc1-43d2-90f9-f2be2b7f3867 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.587947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.588619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.589024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.589490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.590384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.593073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.614794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.614794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.626850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dfba7-4bb2-4f89-8f75-d56336766f59 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.633680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1754/3463] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:17.637305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.653114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.654267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200178c-469f-4fb9-8142-51e602cf33ba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.654781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.655131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.655475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] PATH_INFO: `/v3/projects/8c875b2746384d5a9d02548faab6fd5d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.657801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c875b2746384d5a9d02548faab6fd5d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.660100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.660795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.661472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.661829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.662218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.663196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.665027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.682465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1755/3464] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:17.684095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:17.697574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17a98-1c5c-4db9-9d57-d61d11073423 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:17.703950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:17.704703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.705051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.705512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] PATH_INFO: `/v3/projects/212a64f6557143739a2f3d232127ccbf/users/4a0e7b0511294f0d95ee52d8dacd6446/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.737052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212a64f6557143739a2f3d232127ccbf, user_id=4a0e7b0511294f0d95ee52d8dacd6446, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:17.740568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aa36b-202e-47aa-b758-e04bc9735b84 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:17.758576 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1756/3465] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:01:17 2022] PUT /identity/v3/projects/212a64f6557143739a2f3d232127ccbf/users/4a0e7b0511294f0d95ee52d8dacd6446/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:17.765649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c875b2746384d5a9d02548faab6fd5d', 'request_id': 'req-b849b957-d5a6-4972-b4df-bfd61c182820'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.779054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c875b2746384d5a9d02548faab6fd5d', 'request_id': 'req-b849b957-d5a6-4972-b4df-bfd61c182820'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.779273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.779491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.779775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.784746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c875b2746384d5a9d02548faab6fd5d', 'request_id': 'req-b849b957-d5a6-4972-b4df-bfd61c182820'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.800469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] The token cache is being invalidate because project 8c875b2746384d5a9d02548faab6fd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:01:17.800469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.801858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b849b957-d5a6-4972-b4df-bfd61c182820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:17.807738 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1710/3466] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:01:17 2022] DELETE /identity/v3/projects/8c875b2746384d5a9d02548faab6fd5d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:01:17.809296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa223-3aca-420a-8daa-44a7d2e22cb6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.809673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa223-3aca-420a-8daa-44a7d2e22cb6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.810019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fa223-3aca-420a-8daa-44a7d2e22cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.828611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] MFA Rules not processed for user `4a0e7b0511294f0d95ee52d8dacd6446`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:17.829844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.839635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d52fa223-3aca-420a-8daa-44a7d2e22cb6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:17.841185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d52fa223-3aca-420a-8daa-44a7d2e22cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.930476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1711/3467] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:17.934542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1757/3468] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:17.939600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c33c-47fd-4bc0-ae0e-d3971e39108a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:17.939891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c33c-47fd-4bc0-ae0e-d3971e39108a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:17.940171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c33c-47fd-4bc0-ae0e-d3971e39108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:17.974361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d826c33c-47fd-4bc0-ae0e-d3971e39108a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:17.975821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d826c33c-47fd-4bc0-ae0e-d3971e39108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:17.985984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1712/3469] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:17.990020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.004551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e0873-b6de-4b84-a0e5-07cde86d303e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.011038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.012488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.013865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.016044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.022824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:01:18.024006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.026179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.036296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.036615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.036893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.058687 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1758/3470] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:01:18.062907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.076811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bcc9a-a6d1-43d2-a407-8eb9df38d403 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.083377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:18.085046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:18.088760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.090239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.092271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.092567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.093221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.098996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:18.113485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.133368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:18.174584 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1759/3471] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:01:18 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:01:18.177039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.195939 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1713/3472] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:18.239496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.242213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:18.276031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11193fc6-ef3e-420b-8542-5aef6d7e3352 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.282829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.283612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.283904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.284405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.285489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.287181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.318313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1760/3473] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:01:18 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:01:18.326154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.345688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54ae2-e659-4bf4-b2b8-c93967ab36f7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.350697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.351400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.351581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.351832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.352467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.354070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.366553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1761/3474] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:18.369551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.379436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc057-1dd4-47ea-b367-5d5983affcf1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.379581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dcbb6-ad9f-413d-b1fa-cd207febcb14 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.384043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.384711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.384927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.385197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.385828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.387551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.403078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1762/3475] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:01:18.404896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.416817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18a13-7851-46ef-8b34-8d8c739328bb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.423042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.423815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.424098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.424376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] PATH_INFO: `/v3/projects/1408039f0bcf4130a11bedb2206b3cec/users/5e9e57183b3b49c1ba39b5503420b93e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.443419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1408039f0bcf4130a11bedb2206b3cec, user_id=5e9e57183b3b49c1ba39b5503420b93e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.446072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a178566-38dc-47f8-8bac-0c5ced013cf7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.462139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1763/3476] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:01:18 2022] PUT /identity/v3/projects/1408039f0bcf4130a11bedb2206b3cec/users/5e9e57183b3b49c1ba39b5503420b93e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:18.468221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3717532-136a-4215-a213-0123ed89a6b8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.472355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.473047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.473396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.473677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.474855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3717532-136a-4215-a213-0123ed89a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.475358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3717532-136a-4215-a213-0123ed89a6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.497290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d3717532-136a-4215-a213-0123ed89a6b8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:18.498820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d3717532-136a-4215-a213-0123ed89a6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:18.552703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:18.559028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.561394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.571553 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1714/3477] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:01:18.581545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1764/3478] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:01:18.590887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:18.592806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:18.692060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a3a2-03de-4d9e-8830-4526e7bd08b2 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:18.754249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:18.754932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:18.755211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:18.755479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:18.766183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:18.767954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3510309e-0e18-4cd8-b578-65f81df011ed service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:18.777809 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1715/3479] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:19.183595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e875-0107-4d4b-a912-583953290c6e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:19.183899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e875-0107-4d4b-a912-583953290c6e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:19.184156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72e875-0107-4d4b-a912-583953290c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:19.204594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9f72e875-0107-4d4b-a912-583953290c6e None None] MFA Rules not processed for user `c7c49bed91bb437087a8ad2d865c2647`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:19.205902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f72e875-0107-4d4b-a912-583953290c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:19.232941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:19.264807 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1765/3480] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:28.773145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=49 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:29.058270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c237f-687e-4a74-97d8-bcc4fdaa67d4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:29.058579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c237f-687e-4a74-97d8-bcc4fdaa67d4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:29.059020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c237f-687e-4a74-97d8-bcc4fdaa67d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:29.109775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a39c237f-687e-4a74-97d8-bcc4fdaa67d4 None None] MFA Rules not processed for user `5e9e57183b3b49c1ba39b5503420b93e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:29.111412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a39c237f-687e-4a74-97d8-bcc4fdaa67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:29.239980 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1716/3481] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:29.251818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:29.252383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:29.252383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:29.256439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=11 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:29.256579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:29.257240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:29.257240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:29.296082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:29.297585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] MFA Rules not processed for user `5e9e57183b3b49c1ba39b5503420b93e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:29.299325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:29.302621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:29.316521 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1717/3482] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:29.394450 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1766/3483] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:29.426293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:29.428279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:29.544396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827268a7-91d2-49c7-aeba-6f2983668393 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:29.550332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:29.551037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:29.551483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:29.551623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:29.561899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:29.563664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:29.582923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1718/3484] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:01:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:01:31.212202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:31.213700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:31.408842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3b983-735c-4254-ad5d-752ce3c02acc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:31.427570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:31.428903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:31.429201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:31.429476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:31.452350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:31.454167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955240ee-3a0c-4dcd-a318-c506267a6e9c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:31.482904 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1767/3485] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:37.595411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:37.598478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:37.648670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109a967-42d2-4b07-bd0b-966f4fe42a64 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:37.655688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:37.657116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:37.659466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:37.659981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] PATH_INFO: `/v3/users/4a0e7b0511294f0d95ee52d8dacd6446` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:37.661468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0e7b0511294f0d95ee52d8dacd6446)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:37.664390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:37.765475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0e7b0511294f0d95ee52d8dacd6446'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:37.766965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0e7b0511294f0d95ee52d8dacd6446'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:37.780678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0e7b0511294f0d95ee52d8dacd6446'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:37.792819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0e7b0511294f0d95ee52d8dacd6446'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:37.793628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0e7b0511294f0d95ee52d8dacd6446'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:37.805222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:37.831358 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1719/3486] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:01:37 2022] DELETE /identity/v3/users/4a0e7b0511294f0d95ee52d8dacd6446 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:37.835858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:37.836889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:37.837343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:37.866611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:37.868263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:38.006603 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1768/3487] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:01:38.024361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:38.026077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:38.215255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dd2da-a7bc-4185-a7f2-8cc3c7f896c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:38.220468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:38.221098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:38.221389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:38.222271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:38.253027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:38.254645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea8d66-780a-463b-904b-773481c5b29d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:38.306057 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1720/3488] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:40.491324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:40.493033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:40.509198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd824d79-d7d8-4bb6-913b-96b3e913d3af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:40.533678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:40.535582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:40.536093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:40.537028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] PATH_INFO: `/v3/projects/212a64f6557143739a2f3d232127ccbf` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:40.555461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212a64f6557143739a2f3d232127ccbf)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:40.560360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:40.686594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:40.743979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212a64f6557143739a2f3d232127ccbf', 'request_id': 'req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:40.759660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212a64f6557143739a2f3d232127ccbf', 'request_id': 'req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:40.775616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212a64f6557143739a2f3d232127ccbf', 'request_id': 'req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:40.818719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] The token cache is being invalidate because project 212a64f6557143739a2f3d232127ccbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:01:40.820092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:40.820948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9b99f1b-24e5-4028-a338-d7b0ce0ba941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:40.833352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:40.833905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:40.834341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:40.837812 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1769/3489] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:01:40 2022] DELETE /identity/v3/projects/212a64f6557143739a2f3d232127ccbf => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:40.900386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:40.901818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:41.047806 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1721/3490] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:41.056131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.056477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.056683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.089492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:41.091052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:41.114877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.120168 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1770/3491] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:41.133747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037b049-d304-4bd9-9e1c-7223a418fd97 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.156411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.158411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.158411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.158411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.159002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:01:41.159308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.161506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.202775 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1722/3492] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:01:41.202996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.220630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eedbe-3203-41a9-bbf6-c8070d2217cf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.227473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.228480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.229001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.230269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.231881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.234143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.368605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1771/3493] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:01:41 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:01:41.369778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.404999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdbd72-5921-425c-b67c-df11060ebf35 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.575773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.576578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.577469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.579969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.580816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.583143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.625004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1723/3494] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:01:41 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:01:41.628814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.645514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488c79f-ae9f-4166-92f4-454aac66d81e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.660639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.662032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.662403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.662815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.664147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.666694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.697027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1772/3495] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:41.701285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.712172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:41.721492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf306d2-1aae-4230-9272-b75c236663a3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.738926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.739600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.739877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.740428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.741592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.744344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ab85d-1afe-4317-88ff-e442eebc8203 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.773844 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1724/3496] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:41.778054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:41.788990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66203cbf-fbfa-4af6-8794-268f6b6d63bf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:41.795391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:41.796087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.796325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.796596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] PATH_INFO: `/v3/projects/dd4ce6fc820e4fb5b5e731650ed6faad/users/c70045b8fe87481b8fae4597b979b046/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.834177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4ce6fc820e4fb5b5e731650ed6faad, user_id=c70045b8fe87481b8fae4597b979b046, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:41.837640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:41.862071 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1773/3497] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:01:41 2022] PUT /identity/v3/projects/dd4ce6fc820e4fb5b5e731650ed6faad/users/c70045b8fe87481b8fae4597b979b046/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:41.864065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa547f-057c-4ace-889f-97e0bae1d788 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:41.864558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa547f-057c-4ace-889f-97e0bae1d788 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:41.864640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa547f-057c-4ace-889f-97e0bae1d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:41.904607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-feaa547f-057c-4ace-889f-97e0bae1d788 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:41.907011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-feaa547f-057c-4ace-889f-97e0bae1d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:42.048449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1725/3498] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:42.059826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:42.063433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:42.159408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d12cc-4276-4d7d-aebd-8ce8076a5bbd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:42.166482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:42.167277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:42.167603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:42.168195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:42.191883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:42.196395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:42.245920 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1774/3499] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:01:51.460126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29816ee4-4f67-4594-9ed0-a9489c464799 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:51.460848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29816ee4-4f67-4594-9ed0-a9489c464799 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:51.461192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29816ee4-4f67-4594-9ed0-a9489c464799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:51.533616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-29816ee4-4f67-4594-9ed0-a9489c464799 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:51.543582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29816ee4-4f67-4594-9ed0-a9489c464799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:51.756354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1726/3500] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:51.780603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:51.783369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:51.974452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:52.001365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d899a-58c0-4bbf-bf37-4ca5026605fd service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:52.009089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:52.009808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:52.010249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:52.010864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:52.033471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:52.035415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076d38b-b927-4fcd-8be5-af1420801774 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:52.077543 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1775/3501] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:01:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:01:53.434355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d47c6-3e68-45bd-8ac3-260be9ed2540 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:53.435438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d47c6-3e68-45bd-8ac3-260be9ed2540 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:53.435872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d47c6-3e68-45bd-8ac3-260be9ed2540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:53.514064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dc7d47c6-3e68-45bd-8ac3-260be9ed2540 None None] MFA Rules not processed for user `c70045b8fe87481b8fae4597b979b046`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:53.515844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dc7d47c6-3e68-45bd-8ac3-260be9ed2540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:53.686970 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1727/3502] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:53.700828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:53.707283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:53.707967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:53.812219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] MFA Rules not processed for user `c70045b8fe87481b8fae4597b979b046`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:53.815347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:53.815639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:53.815919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:53.817441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:53.854546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1776/3503] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:53.865710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:53.867749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:53.895240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1728/3504] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:01:53.919070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:53.936708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b31c4-cd4c-47b2-a5f2-4e70363c838e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:53.944230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:53.945516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:53.945971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:53.946381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:53.963971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:53.966697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57dad7-d5d0-4368-8531-6dd43fe2a896 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:53.981630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1777/3505] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:01:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:01:54.260087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:54.261697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:54.321600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:54.463065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3dd48-c073-4e50-9795-1e605065e902 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:54.487842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:54.487842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:54.487842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:54.487842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:54.503980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:54.505702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:54.539139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1729/3506] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:01:54.650639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:54.650923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:54.651233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:54.696090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:54.697738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:54.708963 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1778/3507] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:54.724309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:54.738652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36166f-718a-4420-859a-9bcc8461fc00 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:54.744823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:54.745813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:54.748340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:54.748846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:54.758983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:54.762488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:54.775402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1730/3508] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:01:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:01:55.611338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:55.613399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:55.665559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab11b7-9eae-4916-8a5b-06a1be1d3691 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:55.671443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:55.672549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:55.672997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:55.673483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] PATH_INFO: `/v3/users/5ddab9c74c004e74a51fd8008886e542` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:55.674996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddab9c74c004e74a51fd8008886e542)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:55.678076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:55.771031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddab9c74c004e74a51fd8008886e542'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:55.772542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddab9c74c004e74a51fd8008886e542'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:55.787346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddab9c74c004e74a51fd8008886e542'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:55.804377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddab9c74c004e74a51fd8008886e542'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:55.805959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7cdc116d-5fc4-4888-b7bf-341adf32b2b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddab9c74c004e74a51fd8008886e542'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:55.830964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1779/3509] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:01:55 2022] DELETE /identity/v3/users/5ddab9c74c004e74a51fd8008886e542 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:56.562671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:56.581255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea410d-b47f-474b-9d81-b661a80573c7 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:56.737060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:56.737715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:56.738559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:56.739031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] PATH_INFO: `/v3/projects/981fdf8ae7524612b9c2f6a3f16c39ec` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:56.741528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=981fdf8ae7524612b9c2f6a3f16c39ec)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:56.744499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:56.858443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981fdf8ae7524612b9c2f6a3f16c39ec', 'request_id': 'req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:56.867278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981fdf8ae7524612b9c2f6a3f16c39ec', 'request_id': 'req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:56.880368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981fdf8ae7524612b9c2f6a3f16c39ec', 'request_id': 'req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:56.924650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] The token cache is being invalidate because project 981fdf8ae7524612b9c2f6a3f16c39ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:01:56.924965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:56.926115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d532a26a-7fde-4c5d-abd6-99cd0bbfdd04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:01:56.927542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1731/3510] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:01:56 2022] DELETE /identity/v3/projects/981fdf8ae7524612b9c2f6a3f16c39ec => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:56.935441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:56.935997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:56.936451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:56.975991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:56.979209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:57.086256 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1780/3511] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:57.094056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.095476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.095811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.124728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:57.126344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:57.139239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1732/3512] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:57.143420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.155524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0e591-2f2e-434a-b1b8-0eea49e5f080 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.177766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.178396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.178702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.179329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.180138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:01:57.180620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.184575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.202533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1781/3513] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:01:57.206812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.232430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f405-ef9f-4a10-b15e-c2f61cf712b0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.240204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.240838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.241189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.241825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.242152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.244248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.264468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:57.324317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1733/3514] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:01:57 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:01:57.325482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.343613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6d2f5-625d-41b1-a202-a9629409f626 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.373797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:01:57.469031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.469941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.470330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.470738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.471463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.477091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.523749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1782/3515] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:01:57 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:01:57.525274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.536718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b9c9b-af7c-4eb7-ad2d-571117d67d1f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.548609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.549235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.549514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.549787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.550435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.552085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.568574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1734/3516] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:57.572251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.584224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7eab50-2956-46c1-a39a-bf794ba3492b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.594695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.596046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.596046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.596046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.596670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.598701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c32a7-151a-45b5-893a-d3843f22ab0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.613369 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1783/3517] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:01:57.617172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.637585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8817fd-0190-41a1-9331-daa67ef4e3ea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:57.651151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:57.654337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.654725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.655008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] PATH_INFO: `/v3/projects/5e3dbfe14ac3469286aa595d46c28f56/users/cea166f15a894cd7b1cee6c5a749679c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.672914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3dbfe14ac3469286aa595d46c28f56, user_id=cea166f15a894cd7b1cee6c5a749679c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:57.676619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:57.726584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1735/3518] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:01:57 2022] PUT /identity/v3/projects/5e3dbfe14ac3469286aa595d46c28f56/users/cea166f15a894cd7b1cee6c5a749679c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:01:57.728863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:57.729253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:57.729562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:57.770125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:01:57.772401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:57.898452 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1784/3519] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:01:57.911253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:01:57.914967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:01:58.108392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735caa-d3cb-49ec-8770-94a86d286c19 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:01:58.122694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:01:58.123752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:01:58.124366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:01:58.124789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:01:58.150036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:01:58.153118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:01:58.187061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1736/3520] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:04.576659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:04.578215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:04.606920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cef06-0413-466a-b1a5-9ae926596856 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:04.637846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:04.638583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:04.638861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:04.639108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] PATH_INFO: `/v3/users/c7c49bed91bb437087a8ad2d865c2647` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:04.640391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c49bed91bb437087a8ad2d865c2647)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:04.642336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:04.762387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c49bed91bb437087a8ad2d865c2647'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:04.763060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c49bed91bb437087a8ad2d865c2647'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:04.786069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c49bed91bb437087a8ad2d865c2647'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:04.819527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c49bed91bb437087a8ad2d865c2647'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:04.820335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c49bed91bb437087a8ad2d865c2647'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:04.845892 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1785/3521] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:02:04 2022] DELETE /identity/v3/users/c7c49bed91bb437087a8ad2d865c2647 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:05.614393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:05.631558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83c92c-7475-4a58-b86d-f4b104f8e8b2 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:05.799730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:05.801069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:05.801739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:05.802692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] PATH_INFO: `/v3/projects/5b68f292633a49ca82fb05d18ea83142` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:05.807094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b68f292633a49ca82fb05d18ea83142)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:05.813810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:05.982416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b68f292633a49ca82fb05d18ea83142', 'request_id': 'req-36582725-86b2-4c79-82db-834c21d2c6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.020201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b68f292633a49ca82fb05d18ea83142', 'request_id': 'req-36582725-86b2-4c79-82db-834c21d2c6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.041288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b68f292633a49ca82fb05d18ea83142', 'request_id': 'req-36582725-86b2-4c79-82db-834c21d2c6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.045260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:06.054267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.057043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.079147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] The token cache is being invalidate because project 5b68f292633a49ca82fb05d18ea83142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:02:06.079537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.081940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-36582725-86b2-4c79-82db-834c21d2c6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.082308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf3b6f-5622-4f43-a434-1428737554fc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.086829 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1737/3522] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:02:05 2022] DELETE /identity/v3/projects/5b68f292633a49ca82fb05d18ea83142 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:06.089247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:06.092478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b0e2-4c34-43fd-83fd-16dc6ac5bb3e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.092874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b0e2-4c34-43fd-83fd-16dc6ac5bb3e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.093276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583b0e2-4c34-43fd-83fd-16dc6ac5bb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.137924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c583b0e2-4c34-43fd-83fd-16dc6ac5bb3e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:06.141397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c583b0e2-4c34-43fd-83fd-16dc6ac5bb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.246691 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1738/3523] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:06.254094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.254591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.254719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.257188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.257854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.258402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.258708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] PATH_INFO: `/v3/users/5e9e57183b3b49c1ba39b5503420b93e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.261092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9e57183b3b49c1ba39b5503420b93e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.264281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.296500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:06.298846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.319906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1739/3524] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:06.325019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.345073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f969da-d0ed-4dfa-b57c-4f0627659eae None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.356333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.357348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.357766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.358215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.360527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:02:06.361020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.363680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.382777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1740/3525] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:02:06.387571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.398113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d90f33-30a4-4053-9164-d9f31a7d20f2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.399245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9e57183b3b49c1ba39b5503420b93e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.400538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9e57183b3b49c1ba39b5503420b93e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.404812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.405857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.406234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.406644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.407378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.410526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.416167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9e57183b3b49c1ba39b5503420b93e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.436416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9e57183b3b49c1ba39b5503420b93e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.437775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9e57183b3b49c1ba39b5503420b93e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.450541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1741/3526] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:06.453549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.454157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1786/3527] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:02:06 2022] DELETE /identity/v3/users/5e9e57183b3b49c1ba39b5503420b93e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:02:06.463520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.551978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f2866-8629-4aae-a66f-063b649c6d16 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.628151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.628916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.629308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.629677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.630788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.635626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.644305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:06.673826 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1742/3528] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:06.678546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.690391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5160296-8688-451f-a6f3-0ceb2f663ef5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.695626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.696354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.696837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.697234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.697949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.699883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.704844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.706949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.717840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1787/3529] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:06.722197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.726807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7615693-79d9-4f7b-9ee7-695e8bb33a9e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.735129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.735756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.736271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.736514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63181a1d-2e26-4d4d-896d-b9ba654d69d3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.736641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] PATH_INFO: `/v3/projects/1408039f0bcf4130a11bedb2206b3cec` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.738081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1408039f0bcf4130a11bedb2206b3cec)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.740748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.743402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.744379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.744865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.745206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.746779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.749617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.766200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1788/3530] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:02:06.767360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.777950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a64b4-4a02-4ff2-a05f-cd67180b3f0a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:06.783876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:06.784624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.785006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.785314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] PATH_INFO: `/v3/projects/2ea87ff9b7b54a44a806283f008bd3d1/users/a0815c3afecb4fd5b23a0ee395f7aa5e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.802344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea87ff9b7b54a44a806283f008bd3d1, user_id=a0815c3afecb4fd5b23a0ee395f7aa5e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:06.805189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de15f31-5085-49bc-a0c1-22541e0640ae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:06.816860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1408039f0bcf4130a11bedb2206b3cec', 'request_id': 'req-34f1a571-3593-46ef-9d8e-6e0921d57045'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.823397 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1789/3531] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:02:06 2022] PUT /identity/v3/projects/2ea87ff9b7b54a44a806283f008bd3d1/users/a0815c3afecb4fd5b23a0ee395f7aa5e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:06.828228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1408039f0bcf4130a11bedb2206b3cec', 'request_id': 'req-34f1a571-3593-46ef-9d8e-6e0921d57045'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.829179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.830096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.830313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.840370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1408039f0bcf4130a11bedb2206b3cec', 'request_id': 'req-34f1a571-3593-46ef-9d8e-6e0921d57045'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.851622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:06.853398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.856434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] The token cache is being invalidate because project 1408039f0bcf4130a11bedb2206b3cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:02:06.856800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.857551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34f1a571-3593-46ef-9d8e-6e0921d57045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:06.860006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1743/3532] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:02:06 2022] DELETE /identity/v3/projects/1408039f0bcf4130a11bedb2206b3cec => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:02:06.864217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9d9ba-2c53-44a0-bbfc-702226544bf9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.864434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9d9ba-2c53-44a0-bbfc-702226544bf9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.864823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9d9ba-2c53-44a0-bbfc-702226544bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:06.891154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ffb9d9ba-2c53-44a0-bbfc-702226544bf9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:06.892506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ffb9d9ba-2c53-44a0-bbfc-702226544bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.900611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:06.933039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1790/3533] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:06.947402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:06.950937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:06.986843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1744/3534] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:06.994718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e5670-e5c7-4c4d-9d4c-e52eade43980 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:06.995087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e5670-e5c7-4c4d-9d4c-e52eade43980 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:06.995461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e5670-e5c7-4c4d-9d4c-e52eade43980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.021345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d79e5670-e5c7-4c4d-9d4c-e52eade43980 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:07.023456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d79e5670-e5c7-4c4d-9d4c-e52eade43980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:07.036303 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1745/3535] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:07.037668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.037958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.038232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.041598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871e89-74d5-45b1-bbaf-35fd5bd798af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.045924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.046727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.047108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.047482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.057956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.060712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.064934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:07.066358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:07.073159 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1791/3536] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:07.074763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.085299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676fa1e-f13f-41cc-8b76-533d5dace41d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.090201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.091023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.091389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.091730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.092444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:02:07.092817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.094836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.108858 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1792/3537] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:02:07.113395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.133420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5c04-a68e-4343-96a7-f684e572bacd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.142472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.149201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.149201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.149201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.149201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.149201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.163804 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1746/3538] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:07.198196 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1793/3539] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:02:07 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:02:07.202649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.216981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b179a5-3d30-47c0-a7f1-7ac7e0cf27a9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.227701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:07.291025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.291718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.291999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.293252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.293547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.295889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.330031 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1747/3540] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:02:07 2022] POST /identity/v3/users => generated 286 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:07.331220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.343740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b66b8-95ad-4d6f-a430-eeeb56a9cb71 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.351396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.352023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.352299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.352855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.353699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.354839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.370209 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1794/3541] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:07.374548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.385155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c963c2-aa34-4ce5-8e2b-ac1ff3388b5e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.391027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.391734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.392054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.392328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.393014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.394682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62db79c-8dc9-4b2e-908b-3cccedfa0ced admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.407010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1748/3542] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:07.411757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.424992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9ae3b-141d-4437-9c10-07c54d446de4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.429902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.430604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.430890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.431166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] PATH_INFO: `/v3/projects/6af0aa4ac07b40baa00186abc14ee5a1/users/7c9d24e7a7a94cbd9f235519ac791361/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.448617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af0aa4ac07b40baa00186abc14ee5a1, user_id=7c9d24e7a7a94cbd9f235519ac791361, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.452181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.473067 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1795/3543] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:02:07 2022] PUT /identity/v3/projects/6af0aa4ac07b40baa00186abc14ee5a1/users/7c9d24e7a7a94cbd9f235519ac791361/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:02:07.476240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.476531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.476810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.497692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:07.498955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:07.586334 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1749/3544] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:07.680129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:07.684265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:07.756462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e2697-5feb-400d-be40-dcbb1aaa0969 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:07.760189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:07.768335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:07.769828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:07.770310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:07.804238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:07.811605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:07.820588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1796/3545] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:10.739619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:10.741114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:10.753946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247c8ca-5e1d-4ef5-b0d6-95b0fe61a42d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:10.758812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:10.760476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:10.760769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:10.761047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] PATH_INFO: `/v3/users/f41f618458eb48f6bb4631fd254addc0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:10.762361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41f618458eb48f6bb4631fd254addc0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:10.764611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:10.843218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41f618458eb48f6bb4631fd254addc0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:10.845833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41f618458eb48f6bb4631fd254addc0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:10.860311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41f618458eb48f6bb4631fd254addc0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:10.871205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41f618458eb48f6bb4631fd254addc0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:10.872359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e0a69b9-daff-4451-8b4b-0679935cdb0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41f618458eb48f6bb4631fd254addc0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:10.897948 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1750/3546] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:02:10 2022] DELETE /identity/v3/users/f41f618458eb48f6bb4631fd254addc0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:11.595903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:11.607304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddac4-08fe-4648-aa8b-be475a5bb1e6 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:11.621326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:11.689086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:11.690091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:11.690381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:11.690700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] PATH_INFO: `/v3/projects/d4655b536ca04896993223f34d172298` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:11.691893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4655b536ca04896993223f34d172298)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:11.693913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:11.754455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4655b536ca04896993223f34d172298', 'request_id': 'req-23d83e95-62b7-42c0-98b0-ea6fc03799c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:11.761892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4655b536ca04896993223f34d172298', 'request_id': 'req-23d83e95-62b7-42c0-98b0-ea6fc03799c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:11.770128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4655b536ca04896993223f34d172298', 'request_id': 'req-23d83e95-62b7-42c0-98b0-ea6fc03799c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:11.789977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] The token cache is being invalidate because project d4655b536ca04896993223f34d172298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:02:11.790426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:11.791316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-23d83e95-62b7-42c0-98b0-ea6fc03799c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:11.793457 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1797/3547] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:02:11 2022] DELETE /identity/v3/projects/d4655b536ca04896993223f34d172298 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:11.798480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:11.799669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:11.800164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:11.812711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:11.842901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:11.846562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:11.977642 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1751/3548] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:11.987499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:11.987970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:11.988349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.046658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:12.049449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:12.066125 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1798/3549] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:12.070351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.084860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ecfd1-92ca-4c85-85b9-05291cd1c58c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.111892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.115064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.117012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.146999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.148421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:02:12.149555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.151700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.193107 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1752/3550] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:02:12.195544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.210528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a5ef3-8b22-466c-a60b-f1e86ac2b0e0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.242460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.243189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.247750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.250260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.251690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.264299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.329295 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1799/3551] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:02:12 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:12.334039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.334206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.336185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:12.485643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bc274-b755-4c0f-b37b-6d4d960f4df8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.492407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ad8e-6192-4ee9-9b14-5b506813f047 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.500473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.502791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.503221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.503498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.504506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.511623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.569942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1800/3552] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:02:12 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:12.571636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.603044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e51cb-d760-4445-befb-75f2ecf2567e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.616985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.618933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.619375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.620070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.623306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.625144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.630991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:12.639236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.639864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.640146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.640488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.647498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:12.663817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1801/3553] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:12.664935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.679715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578e30f-ac42-4003-bac5-6e97774b7c70 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.687553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.688599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.689044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.689464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.691550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.694112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.713479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1802/3554] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:12.717924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:12.726665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:12.732880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35dd35-ab4d-4e14-aa0c-2471f3c23f3a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:12.740617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:12.741813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.742298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.742755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] PATH_INFO: `/v3/projects/fd99c2803f734c47b8832a274bf1a5a0/users/9853254b84cc425c939eb166e62db87a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.770460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd99c2803f734c47b8832a274bf1a5a0, user_id=9853254b84cc425c939eb166e62db87a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.773593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6b955-a257-4a8c-9656-0db2ad3c7521 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.799866 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1803/3555] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:02:12 2022] PUT /identity/v3/projects/fd99c2803f734c47b8832a274bf1a5a0/users/9853254b84cc425c939eb166e62db87a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:12.801778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152713-7e48-4b3a-bd76-18cfbfa0d332 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:12.802219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152713-7e48-4b3a-bd76-18cfbfa0d332 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:12.802695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152713-7e48-4b3a-bd76-18cfbfa0d332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:12.842156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-66152713-7e48-4b3a-bd76-18cfbfa0d332 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:12.846432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-66152713-7e48-4b3a-bd76-18cfbfa0d332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:12.863322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:12.867265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:12.912851 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1753/3556] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:13.019069 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1804/3557] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:13.076061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:13.080639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:13.238880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fee1de-2680-40de-b612-67940dcec8d8 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:13.350039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:13.350694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:13.350976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:13.351249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:13.362771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:13.364370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:13.376732 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1754/3558] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:18.346453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32231c-3ef5-455f-89bd-5f2c6a558423 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:18.346988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32231c-3ef5-455f-89bd-5f2c6a558423 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:18.347666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32231c-3ef5-455f-89bd-5f2c6a558423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:18.404586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8b32231c-3ef5-455f-89bd-5f2c6a558423 None None] MFA Rules not processed for user `cea166f15a894cd7b1cee6c5a749679c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:18.406785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b32231c-3ef5-455f-89bd-5f2c6a558423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:18.556518 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1805/3559] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:18.573429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:18.576454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:18.712063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:18.758028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42065c-d9a1-408b-bb49-030af97850e7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:18.777313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:18.779285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:18.779285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:18.779285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:18.795502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:18.797641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef9df5-20a9-4c1e-8cff-1ac759d225cc service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:18.830955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1755/3560] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:19.975469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e5a-7666-4c8d-8403-9db1894fe372 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:19.976455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e5a-7666-4c8d-8403-9db1894fe372 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:19.977254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe10e5a-7666-4c8d-8403-9db1894fe372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:20.006680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cfe10e5a-7666-4c8d-8403-9db1894fe372 None None] MFA Rules not processed for user `ee21da3ff38a43798bce50c9aba4ae33`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:20.011510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfe10e5a-7666-4c8d-8403-9db1894fe372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:20.091678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:20.142074 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1806/3561] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:20.227941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:20.227941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:20.228941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:20.343155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:20.356287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:20.566034 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1756/3562] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:20.584916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05e4-8416-4c08-a9ef-71759bac56c3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:20.585654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05e4-8416-4c08-a9ef-71759bac56c3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:20.586025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f05e4-8416-4c08-a9ef-71759bac56c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:20.643152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-037f05e4-8416-4c08-a9ef-71759bac56c3 None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:20.646859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-037f05e4-8416-4c08-a9ef-71759bac56c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:20.676056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1807/3563] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:20.690751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:20.714399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c58c81-6493-4222-9755-08c906af0da0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:20.724555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:20.725603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:20.725985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:20.726358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:20.751628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:20.753416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a46222-838b-40c6-9426-e091887f9a1d service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:20.776038 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1757/3564] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:26.586385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd4ff-1e68-43db-866f-fcca690fdf46 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:26.588401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd4ff-1e68-43db-866f-fcca690fdf46 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:26.589445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efd4ff-1e68-43db-866f-fcca690fdf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:26.697989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-42efd4ff-1e68-43db-866f-fcca690fdf46 None None] MFA Rules not processed for user `9853254b84cc425c939eb166e62db87a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:26.701319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-42efd4ff-1e68-43db-866f-fcca690fdf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:26.762208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:26.762867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:26.762867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:26.835601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] MFA Rules not processed for user `7c9d24e7a7a94cbd9f235519ac791361`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:26.839589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:26.916222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1808/3565] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:27.004321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:27.004946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:27.005404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:27.015119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1758/3566] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:02:27.030239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] MFA Rules not processed for user `7c9d24e7a7a94cbd9f235519ac791361`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:27.031829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:27.045682 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1809/3567] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:02:27.231353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:27.249248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee311b-5c76-4536-a6ab-5868a30f622d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:27.258148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:27.259024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:27.269486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:27.269968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:27.289571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:27.291644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd233c3a-00f3-418e-982f-d2689a0127e5 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:27.316627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:27.322008 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1759/3568] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:27.333722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22843f-317a-40b7-805e-80b6cef92432 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:27.341744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:27.342780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:27.343218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:27.343888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:27.356425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:27.357537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:27.371541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1810/3569] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:02:27.428893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b33a9-e912-47e6-9307-0ea6a9880d33 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:27.429175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b33a9-e912-47e6-9307-0ea6a9880d33 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:27.429454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b33a9-e912-47e6-9307-0ea6a9880d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:27.465599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d46b33a9-e912-47e6-9307-0ea6a9880d33 None None] MFA Rules not processed for user `cea166f15a894cd7b1cee6c5a749679c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:27.467306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d46b33a9-e912-47e6-9307-0ea6a9880d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:27.489518 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1760/3570] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:37.364637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=46 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:37.458309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:41.724064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:41.725588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:41.808759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a09d68-d339-4907-902a-3a4629526d30 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:41.813916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:41.814588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:41.814866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:41.815141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:41.816950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:41.829955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:41.831585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:41.842962 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1811/3571] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:47.123269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:47.124113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:47.124556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:47.150186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:47.151844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:47.161865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1761/3572] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:47.658884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:47.661382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:47.702990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c1aec-54dc-40a5-ad89-8c44342461a3 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:47.716758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:47.717685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:47.718189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:47.718696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] PATH_INFO: `/v3/users/c70045b8fe87481b8fae4597b979b046` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:47.720583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c70045b8fe87481b8fae4597b979b046)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:47.722731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:47.804930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c70045b8fe87481b8fae4597b979b046'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:47.805769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c70045b8fe87481b8fae4597b979b046'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:47.816440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c70045b8fe87481b8fae4597b979b046'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:47.847125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c70045b8fe87481b8fae4597b979b046'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:47.847865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e721aeaa-98b7-47b5-a099-a1985f8bf604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c70045b8fe87481b8fae4597b979b046'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:47.870585 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1812/3573] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:02:47 2022] DELETE /identity/v3/users/c70045b8fe87481b8fae4597b979b046 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:48.090547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.139596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ec4f8-610a-45dd-b84f-f34eeb7f041f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.177634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916e47-e993-46b0-9108-4a3070fc6615 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.178003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916e47-e993-46b0-9108-4a3070fc6615 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.178179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af916e47-e993-46b0-9108-4a3070fc6615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.218229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-af916e47-e993-46b0-9108-4a3070fc6615 None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:48.219673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af916e47-e993-46b0-9108-4a3070fc6615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:48.252217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.253267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.253668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.254049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] PATH_INFO: `/v3/projects/dd4ce6fc820e4fb5b5e731650ed6faad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.256974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4ce6fc820e4fb5b5e731650ed6faad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.259274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.344181 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1813/3574] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:48.350075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4ce6fc820e4fb5b5e731650ed6faad', 'request_id': 'req-03c0f648-cafd-4de6-8c70-790d02e0905a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:48.357955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4ce6fc820e4fb5b5e731650ed6faad', 'request_id': 'req-03c0f648-cafd-4de6-8c70-790d02e0905a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:48.371024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4ce6fc820e4fb5b5e731650ed6faad', 'request_id': 'req-03c0f648-cafd-4de6-8c70-790d02e0905a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:48.386735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] The token cache is being invalidate because project dd4ce6fc820e4fb5b5e731650ed6faad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:02:48.387021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:48.387739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-03c0f648-cafd-4de6-8c70-790d02e0905a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:48.390220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1762/3575] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:02:48 2022] DELETE /identity/v3/projects/dd4ce6fc820e4fb5b5e731650ed6faad => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:48.394391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.394705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.394989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.413200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:48.422121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:48.423735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:48.500141 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1814/3576] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:48.509026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.509467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.509895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.544240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:48.545545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:48.560026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1763/3577] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:48.564504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.579385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f950f-33df-4ef4-a666-08e1419524f4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.584900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.586251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.586863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.587304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.587676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:02:48.588132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.590280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5729d4-5c6c-47d1-bdf8-1cefae54f436 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.612389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1815/3578] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:02:48.614679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.627914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe238f-3efd-4225-a528-3d23eed2fb17 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.637483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.639255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.639650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.640026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.640971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.643295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.698818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.699330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.699453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.702093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1764/3579] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:48.710681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.746006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:48.748196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:48.804066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:48.809153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd98b5-f170-4526-8538-664ddd97acfc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.831397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.832069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.832348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.832617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.833563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.837075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.872745 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1816/3580] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:48.884379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.884587 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1765/3581] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:02:48 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:48.908834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa737e2-ef95-44b0-952e-55c6de33aa2b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.915503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.916997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.917535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.918033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.919106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.921122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.947069 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1817/3582] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:48.948127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:48.967455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb09fd5-7b0e-45cb-88b8-57b666314596 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:48.973339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351606-0488-459b-94a6-39462035814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:48.973955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351606-0488-459b-94a6-39462035814d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:48.974235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351606-0488-459b-94a6-39462035814d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:48.974528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351606-0488-459b-94a6-39462035814d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:48.975588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351606-0488-459b-94a6-39462035814d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:48.978275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351606-0488-459b-94a6-39462035814d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:48.989286 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1766/3583] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:02:48.993516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:49.005410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f9f6e-15ec-4bad-ab8b-a0e03a5bd1b0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:49.016232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:49.018263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.018772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.019146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] PATH_INFO: `/v3/projects/7128b36583514196945b5a65446e3e2c/users/b75065b5a5084b639bdab6e7dce97302/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.042809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7128b36583514196945b5a65446e3e2c, user_id=b75065b5a5084b639bdab6e7dce97302, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:49.048139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d1e45-7457-49a5-835e-6694f2446619 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:49.082942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1818/3584] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:02:48 2022] PUT /identity/v3/projects/7128b36583514196945b5a65446e3e2c/users/b75065b5a5084b639bdab6e7dce97302/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:02:49.097565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9667-8841-4ea4-bd82-ca8a57778b44 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.097858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9667-8841-4ea4-bd82-ca8a57778b44 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.098142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e9667-8841-4ea4-bd82-ca8a57778b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.140373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8d5e9667-8841-4ea4-bd82-ca8a57778b44 None None] MFA Rules not processed for user `b75065b5a5084b639bdab6e7dce97302`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:49.141842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8d5e9667-8841-4ea4-bd82-ca8a57778b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.231654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.231654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.231996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.258391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:49.260326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.279200 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1767/3585] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:49.281230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.281689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.282002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.321191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] MFA Rules not processed for user `b75065b5a5084b639bdab6e7dce97302`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:49.322631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.340398 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1768/3586] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:49.359732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1819/3587] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:49.451460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:49.453932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.574836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e506d9c-4231-4b76-8d82-f1e6cb045d4a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:49.583952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:49.584703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.585053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.585393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.598274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:49.601527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acbc4f-66c8-4d1c-a359-7e8bc38cbd4c service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:49.621542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1769/3588] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:49.683298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:49.685259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.698989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:49.817393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473b8a4-8817-4119-8479-44db9b645da3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:49.829854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:49.830581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.830904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.831178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.833786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.853143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:49.855699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:49.882853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1820/3589] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:02:49.884368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:49.884652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:49.884956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:49.933268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] MFA Rules not processed for user `a0815c3afecb4fd5b23a0ee395f7aa5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:49.936214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:49.951724 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1770/3590] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:57.026862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:57.028325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:57.190431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe8c60-7451-4fd5-a371-7409119487ae service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:57.195313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:57.196413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:57.196851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:57.197396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] PATH_INFO: `/v3/users/9853254b84cc425c939eb166e62db87a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:57.199292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9853254b84cc425c939eb166e62db87a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:57.201285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:57.310710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9853254b84cc425c939eb166e62db87a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:57.311589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9853254b84cc425c939eb166e62db87a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:57.330784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9853254b84cc425c939eb166e62db87a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:57.367085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9853254b84cc425c939eb166e62db87a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:57.368167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7e93c07-90c1-42fa-9920-823a3adcfa3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9853254b84cc425c939eb166e62db87a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:57.406372 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1821/3591] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:02:57 2022] DELETE /identity/v3/users/9853254b84cc425c939eb166e62db87a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:58.079441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.091802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c937874-dfda-4d7b-b8fd-4059c8347615 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.134126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:58.175550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.176209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.176485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.176757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] PATH_INFO: `/v3/projects/fd99c2803f734c47b8832a274bf1a5a0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.182886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd99c2803f734c47b8832a274bf1a5a0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.185082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.237310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd99c2803f734c47b8832a274bf1a5a0', 'request_id': 'req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:58.244457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd99c2803f734c47b8832a274bf1a5a0', 'request_id': 'req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:58.256139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd99c2803f734c47b8832a274bf1a5a0', 'request_id': 'req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:58.268598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] The token cache is being invalidate because project fd99c2803f734c47b8832a274bf1a5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:02:58.268881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:58.270791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6f53f48-6ca8-4fc1-b051-fa8c517ec2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:02:58.273431 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1771/3592] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:02:58 2022] DELETE /identity/v3/projects/fd99c2803f734c47b8832a274bf1a5a0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:02:58.278784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.279311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.279311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.306310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:58.308570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:58.324450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:02:58.398001 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1822/3593] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:02:58.406617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.406909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.407189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.428761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:58.429985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:58.438811 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1772/3594] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:02:58.443014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.453020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d283d-a1dc-4ed0-92ee-1f48f12b117b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.457654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.458273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.458575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.458853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.459524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:02:58.459828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.461429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.483481 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1823/3595] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:02:58.484601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.494791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16eee4c-06e0-449a-b5fe-2583087cf72e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.499761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.500360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.500643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.500918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.501893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.504322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.533079 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1773/3596] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:02:58 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:02:58.537013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.547779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad6280-7fcd-44c7-b935-f598cb3e74b5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.614880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.615500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.615779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.616047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.616866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.618807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.649535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1824/3597] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:02:58 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:02:58.653465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.666477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146d16d-4e5a-4c19-b58d-c6b1fc7f523a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.671852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.672484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.672767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.673033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.673668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.675311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.687377 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1774/3598] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:02:58.691347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.713986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30bdfe-7b06-4ef6-b0b8-558f1f20cc20 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.718813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.719417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.719693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.719961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.720604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.722214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b4f43-1496-497b-8464-c8a3d588bc8f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.733489 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1825/3599] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:02:58.737404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.748651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1f27-1970-4f75-90a1-b6c3aceccdd2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.753473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.754138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.754436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.754728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] PATH_INFO: `/v3/projects/dad3759161984bb896e1423af9a91d14/users/7d8199c7aba04a9fb1212a2c7e1a52b3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.770923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad3759161984bb896e1423af9a91d14, user_id=7d8199c7aba04a9fb1212a2c7e1a52b3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.773582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:58.787098 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1775/3600] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:02:58 2022] PUT /identity/v3/projects/dad3759161984bb896e1423af9a91d14/users/7d8199c7aba04a9fb1212a2c7e1a52b3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:02:58.792287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.792574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.792850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.812570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:02:58.813822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:58.898752 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1826/3601] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:02:58.908943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:02:58.910717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:02:58.973796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca1f8-092a-4151-a07e-41890ffe9f35 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:02:58.978008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:02:58.979253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:02:58.979536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:02:58.979809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:02:58.990389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:02:58.991895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae34be-24cb-44fb-a195-5c9d8099a562 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:02:59.002602 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1776/3602] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:03.932251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:03.934113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:03.955984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18901407-21c1-41c4-9ed6-51edbab716cb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:03.971344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:03.972270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:03.972767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:03.973245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] PATH_INFO: `/v3/users/b75065b5a5084b639bdab6e7dce97302` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:03.975629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75065b5a5084b639bdab6e7dce97302)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:03.981868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:04.062704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:04.153057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75065b5a5084b639bdab6e7dce97302'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:04.157294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75065b5a5084b639bdab6e7dce97302'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:04.169339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75065b5a5084b639bdab6e7dce97302'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:04.194813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75065b5a5084b639bdab6e7dce97302'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:04.197190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75065b5a5084b639bdab6e7dce97302'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:04.229847 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1827/3603] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:03 2022] DELETE /identity/v3/users/b75065b5a5084b639bdab6e7dce97302 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:04.234236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:04.236085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:04.236558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:04.268153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:04.291410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:04.292950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:04.491470 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1777/3604] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:04.519275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:04.521222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:04.624370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84a90b-35f0-42fb-a764-75a2e3a8dc3f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:04.630279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:04.630720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:04.631002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:04.631366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:04.648474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:04.651321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:04.664339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1828/3605] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:05.241414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:05.243110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:05.266286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f9df-e3ab-4265-bf3a-3183e764a930 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:05.298453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:05.299730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:05.300422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:05.301801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] PATH_INFO: `/v3/users/7c9d24e7a7a94cbd9f235519ac791361` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:05.305125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9d24e7a7a94cbd9f235519ac791361)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:05.309825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:05.454040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9d24e7a7a94cbd9f235519ac791361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:05.460802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9d24e7a7a94cbd9f235519ac791361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:05.473350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9d24e7a7a94cbd9f235519ac791361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:05.493648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9d24e7a7a94cbd9f235519ac791361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:05.494456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19905927-2645-4e74-bfff-c287baff5962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9d24e7a7a94cbd9f235519ac791361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:05.526593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1778/3606] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:05 2022] DELETE /identity/v3/users/7c9d24e7a7a94cbd9f235519ac791361 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:05.776100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:05.789368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9525-8710-4102-a968-f95a9610c678 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:05.916648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:05.917705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:05.918201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:05.918739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] PATH_INFO: `/v3/projects/6af0aa4ac07b40baa00186abc14ee5a1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:05.920168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af0aa4ac07b40baa00186abc14ee5a1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:05.922317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:05.997633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:06.026115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af0aa4ac07b40baa00186abc14ee5a1', 'request_id': 'req-108b39a9-efaa-4c60-9093-296326497335'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.033545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af0aa4ac07b40baa00186abc14ee5a1', 'request_id': 'req-108b39a9-efaa-4c60-9093-296326497335'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.041875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af0aa4ac07b40baa00186abc14ee5a1', 'request_id': 'req-108b39a9-efaa-4c60-9093-296326497335'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.065866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] The token cache is being invalidate because project 6af0aa4ac07b40baa00186abc14ee5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:06.066354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.068671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.082879 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1829/3607] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:05 2022] DELETE /identity/v3/projects/6af0aa4ac07b40baa00186abc14ee5a1 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:06.089244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c209-0131-4f52-a191-66d3c3c21af5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.089244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c209-0131-4f52-a191-66d3c3c21af5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.089244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c209-0131-4f52-a191-66d3c3c21af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.135987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-39f7c209-0131-4f52-a191-66d3c3c21af5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:06.138895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-39f7c209-0131-4f52-a191-66d3c3c21af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.161330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.167453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.308138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1779/3608] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:06.316828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.317209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.318157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.326404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b39a9-efaa-4c60-9093-296326497335 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.333737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:06.334402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.335103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.335386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] PATH_INFO: `/v3/projects/7128b36583514196945b5a65446e3e2c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.337329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7128b36583514196945b5a65446e3e2c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:06.340555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:06.344691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:06.349147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.375334 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1780/3609] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:06.378217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.380688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.411823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e865a9f-7738-431a-b2cc-463e81721049 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.420540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7128b36583514196945b5a65446e3e2c', 'request_id': 'req-4cfc523c-bd90-40eb-8774-854f85b46cfe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.433316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:06.435537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7128b36583514196945b5a65446e3e2c', 'request_id': 'req-4cfc523c-bd90-40eb-8774-854f85b46cfe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.445681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7128b36583514196945b5a65446e3e2c', 'request_id': 'req-4cfc523c-bd90-40eb-8774-854f85b46cfe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.476168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] The token cache is being invalidate because project 7128b36583514196945b5a65446e3e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:06.476483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.480394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.487880 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1830/3610] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:06 2022] DELETE /identity/v3/projects/7128b36583514196945b5a65446e3e2c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:06.487880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.491381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.549707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:06.550787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.551073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.551348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] PATH_INFO: `/v3/users/a0815c3afecb4fd5b23a0ee395f7aa5e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.553361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0815c3afecb4fd5b23a0ee395f7aa5e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:06.555412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:06.607419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc523c-bd90-40eb-8774-854f85b46cfe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.613167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:06.613837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.614160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.614450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.615193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:06.615526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:06.617366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7dea2-bd23-4e8f-947e-7220517236aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:06.620907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:06.637236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1831/3611] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:06.638294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.639113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.639384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.646899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0815c3afecb4fd5b23a0ee395f7aa5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.647574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0815c3afecb4fd5b23a0ee395f7aa5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.657399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0815c3afecb4fd5b23a0ee395f7aa5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.670667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0815c3afecb4fd5b23a0ee395f7aa5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.671362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df20e6db-f020-40fe-83fa-75e20ebc2fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0815c3afecb4fd5b23a0ee395f7aa5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:06.671811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:06.673069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.685380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1832/3612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:06.686409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.688605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.697758 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1781/3613] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:06 2022] DELETE /identity/v3/users/a0815c3afecb4fd5b23a0ee395f7aa5e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:06.699298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.699586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.699873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.712416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7e02-7e93-4b13-aeba-0b9ac1742f7a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.727831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:06.730359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:06.811179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:06.813132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.813658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.814195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.815079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:06.818013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:06.860868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1782/3614] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:06.865236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.880550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c6906-a0f1-4ea4-b90d-a6ae97415e76 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.891765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1833/3615] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/projects => generated 367 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:06.892615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:06.894678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:06.895208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:06.895730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:06.896749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:06.897282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:06.897418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.899959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:06.916869 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1783/3616] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:03:06.921509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:06.934226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bcf0-7b2e-4aa4-8137-594348861ecd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:06.951370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:07.016078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.017595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.017595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.018113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.023185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.026277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.071957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eaf46-85b6-40cd-b617-0bc1b585248e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.089190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.091072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.092601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.093673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.094532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.096486 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1784/3617] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:03:07.099355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.100212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.107590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.138006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1834/3618] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:06 2022] POST /identity/v3/users => generated 284 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:07.139339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.151969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a3e41-e31d-4360-818a-7b258325c9fe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.194488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb8452-b50a-4e08-bb5e-239534e73e87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.200430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.201463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.201941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.202408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] PATH_INFO: `/v3/projects/2ea87ff9b7b54a44a806283f008bd3d1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.203983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea87ff9b7b54a44a806283f008bd3d1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.206746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.232786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.232990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.233533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.234755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.235614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.249738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.263525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:07.279864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea87ff9b7b54a44a806283f008bd3d1', 'request_id': 'req-98ca93a9-418a-471a-811a-3d940531e33d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:07.286394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1835/3619] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:07.287552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.291336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea87ff9b7b54a44a806283f008bd3d1', 'request_id': 'req-98ca93a9-418a-471a-811a-3d940531e33d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:07.303234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0ab3-9c10-49fa-b1ca-83d75fcba968 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.321134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea87ff9b7b54a44a806283f008bd3d1', 'request_id': 'req-98ca93a9-418a-471a-811a-3d940531e33d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:07.352709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] The token cache is being invalidate because project 2ea87ff9b7b54a44a806283f008bd3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:07.353467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:07.354569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:07.358832 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1785/3620] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:07 2022] DELETE /identity/v3/projects/2ea87ff9b7b54a44a806283f008bd3d1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:07.359842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.365892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.438677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.440491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.440791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.441071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.441883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.445668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:07.446426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edf305-2050-4bcb-832e-0ffaeac338ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.461780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1836/3621] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:07.463748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.464088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.465060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.497960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:07.499613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.514766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca93a9-418a-471a-811a-3d940531e33d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.529688 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1837/3622] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:07.530795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.533284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.534400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.534400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.534657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.535248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.535371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.537360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989ee65-dd5d-4be8-bc8e-9eace57c805d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.565602 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1786/3623] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:07.566119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812b7bf-a94e-41be-9d08-2cda00f10f3e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.568095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.568095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.573566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.585218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.585843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.592544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.592833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.600304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.601967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.606561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:07.608076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.622638 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1787/3624] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:07.622638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.623057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1838/3625] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:07.623846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.657749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77babc35-d585-470d-961c-b7f7a0f61508 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.659428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c1adf-fd78-4f65-91c1-341cc048aa0f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.668808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.669437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.671397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.672017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.672296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.672571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.673221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.673866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.674195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.674942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:07.675247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.678575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.679808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.706427 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1839/3626] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:07.706652 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1788/3627] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:07.706763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.706848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.724545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63268dfb-8db7-45dd-bf9e-4a91d2d6fdc4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.730102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f3714-88f8-47e2-a599-acbd8d51b17e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.735576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.736855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.737350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.737832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] PATH_INFO: `/v3/projects/accafa2724354ca8b1c9f19940989429/users/e557d91b49c143499d809a55e85932fd/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.739605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:07.740295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.740575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.740853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] PATH_INFO: `/v3/projects/e6d75184d9cd4fed89c0e1d0d5d910fd/users/7654d225081948cabad9d2cdbf03ebda/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.758177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=accafa2724354ca8b1c9f19940989429, user_id=e557d91b49c143499d809a55e85932fd, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.762287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.776485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d75184d9cd4fed89c0e1d0d5d910fd, user_id=7654d225081948cabad9d2cdbf03ebda, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:07.780843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21853643-75cd-4077-b64f-68fde2cf7bb5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:07.784042 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1840/3628] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:07 2022] PUT /identity/v3/projects/accafa2724354ca8b1c9f19940989429/users/e557d91b49c143499d809a55e85932fd/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:07.785241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.801157 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1789/3629] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:07 2022] PUT /identity/v3/projects/e6d75184d9cd4fed89c0e1d0d5d910fd/users/7654d225081948cabad9d2cdbf03ebda/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:07.802614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baabd5c-6974-4b5c-b0d0-94b2bd9b3e85 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.803184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baabd5c-6974-4b5c-b0d0-94b2bd9b3e85 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.803508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baabd5c-6974-4b5c-b0d0-94b2bd9b3e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.835426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3baabd5c-6974-4b5c-b0d0-94b2bd9b3e85 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:07.836729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3baabd5c-6974-4b5c-b0d0-94b2bd9b3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.903164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505a57-e813-484c-9b7e-2e9644d79fa5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:07.934180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1790/3630] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:07.936257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:07.936730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:07.937309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:07.962014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:07.963274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:07.973313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1791/3631] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:07.974008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:07.975118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:07.975514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.001824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.002468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.002827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.003416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.003876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.005941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.043463 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1841/3632] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:07 2022] POST /identity/v3/projects => generated 393 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:08.044493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.051353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed659c0-e088-45cd-9f74-59eb072ffa27 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.099415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:08.122697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.123963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.123963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.124192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.130371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08066-2870-4eb2-94a4-e4420aa334ae admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.137005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.137767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.138322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.138642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.238025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.242839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.262844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.264873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7485c2-9ff3-4193-9c45-a9cec81e0e01 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.268353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1792/3633] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:08.269192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.279616 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1842/3634] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:08.280695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc437-d6d8-4a66-9e9e-a3d8e6044494 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.280846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521bea4-4ad1-4d2e-8af0-fb91fb37268f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.281084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521bea4-4ad1-4d2e-8af0-fb91fb37268f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.281257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521bea4-4ad1-4d2e-8af0-fb91fb37268f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.286327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.287011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.287289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.287558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.288181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.290113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.323525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d521bea4-4ad1-4d2e-8af0-fb91fb37268f None None] MFA Rules not processed for user `7d8199c7aba04a9fb1212a2c7e1a52b3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:08.326240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1793/3635] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:08 2022] POST /identity/v3/users => generated 297 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:08.329346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d521bea4-4ad1-4d2e-8af0-fb91fb37268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.332114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.359222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d5127-6e77-4a17-81a3-07b7c3b8b9d8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.366816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.367946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.368530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.370248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.371364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.373133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.400706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1794/3636] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:08.402606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.416909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74741745-c5bb-4213-b3e1-69cc62349210 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.423984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.424877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.425284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.425649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.426557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.428777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.448928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1795/3637] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:08.449144 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1843/3638] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:08.450556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.457583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.457855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.458138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.467175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:08.484339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206078-b43c-40ed-8f68-99f4b9087338 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.485374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] MFA Rules not processed for user `7d8199c7aba04a9fb1212a2c7e1a52b3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:08.491611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.499934 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1844/3639] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:03:08.500885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.501501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.501785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.502053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] PATH_INFO: `/v3/projects/3a9bed43edcf4c5594ffde33282bc48c/users/5cb9430023d842b4a5d930e01f7340f3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.511127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.513240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.542618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9bed43edcf4c5594ffde33282bc48c, user_id=5cb9430023d842b4a5d930e01f7340f3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.547569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d251cf1-a970-49fc-8835-ee89f2aa5b62 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.575627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1796/3640] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:08 2022] PUT /identity/v3/projects/3a9bed43edcf4c5594ffde33282bc48c/users/5cb9430023d842b4a5d930e01f7340f3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:08.580793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.581448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.582103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.626970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:08.628751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.682694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62472d82-1bac-4d2a-8d0e-d7a4a6c1bc3e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.758152 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1797/3641] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:03:08.792948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.793577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.793859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.794136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.861652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:08.864673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:08.902589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:08.905603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d515cc-9313-474c-8ddc-992e6c4fef92 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:08.926918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:08.932341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1845/3642] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:08.977323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13867eb0-e631-4242-99e7-ab6a1bbb6ef9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:08.984001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:08.985375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:08.985851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:08.986345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:08.997682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:09.003542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:09.027003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1798/3643] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:09.830200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:09.830461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:09.830924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:09.879722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:09.881128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:09.932783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:09.995579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1846/3644] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:11.251402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:11.262260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86414152-b825-4bba-b1f3-034b0b904f97 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:11.271718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:11.272521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:11.272810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:11.273087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:11.283636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:11.311652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:11.313304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecfb70-2660-4e65-8449-c73dddee21c7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:11.335940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1799/3645] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:19.984175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:20.091244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:20.093612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:20.114437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afcbe-8e39-4e90-ba37-b5c764171fb4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:20.123299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:20.124572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:20.126045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:20.126335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] PATH_INFO: `/v3/users/ee21da3ff38a43798bce50c9aba4ae33` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:20.128473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee21da3ff38a43798bce50c9aba4ae33)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:20.131088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:20.235501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee21da3ff38a43798bce50c9aba4ae33'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:20.237672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee21da3ff38a43798bce50c9aba4ae33'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:20.274009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee21da3ff38a43798bce50c9aba4ae33'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:20.301060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee21da3ff38a43798bce50c9aba4ae33'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:20.303058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee21da3ff38a43798bce50c9aba4ae33'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:20.348220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1847/3646] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:20 2022] DELETE /identity/v3/users/ee21da3ff38a43798bce50c9aba4ae33 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:21.329485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=21 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:22.037048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319162-30c9-4f27-90de-267d9380381d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.039984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319162-30c9-4f27-90de-267d9380381d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.041101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319162-30c9-4f27-90de-267d9380381d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.085670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:22.089753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.095361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-38319162-30c9-4f27-90de-267d9380381d None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:22.097277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-38319162-30c9-4f27-90de-267d9380381d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.274107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fa635-e339-4bfc-a326-f7e4a6866fb9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:22.281361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:22.282299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.283213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.283727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] PATH_INFO: `/v3/projects/8340f7f8fa9848dc822903c738f78f2a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.285441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8340f7f8fa9848dc822903c738f78f2a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:22.287700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:22.328219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1800/3647] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:22.338973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292475c-465b-4070-9f91-9aa70186f33a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.341126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292475c-465b-4070-9f91-9aa70186f33a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.348579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292475c-465b-4070-9f91-9aa70186f33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.368449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8340f7f8fa9848dc822903c738f78f2a', 'request_id': 'req-a6068324-7c48-462d-a1bb-9f3f8d0ce579'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:22.379956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8340f7f8fa9848dc822903c738f78f2a', 'request_id': 'req-a6068324-7c48-462d-a1bb-9f3f8d0ce579'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:22.390992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8340f7f8fa9848dc822903c738f78f2a', 'request_id': 'req-a6068324-7c48-462d-a1bb-9f3f8d0ce579'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:22.396104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d292475c-465b-4070-9f91-9aa70186f33a None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:22.399914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d292475c-465b-4070-9f91-9aa70186f33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.416776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] The token cache is being invalidate because project 8340f7f8fa9848dc822903c738f78f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:22.417387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:22.418130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a6068324-7c48-462d-a1bb-9f3f8d0ce579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:22.420622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1848/3648] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:22 2022] DELETE /identity/v3/projects/8340f7f8fa9848dc822903c738f78f2a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:22.430268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.431808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.432273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.456638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:22.499099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:22.509651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.581635 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1801/3649] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:22.727964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1849/3650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:22.730177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.730463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.730776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.782201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:22.784565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.825335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1802/3651] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:22.829491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:22.850195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c6eb-2062-4828-ae15-f008b61e1afd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:22.870474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:22.871261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.871643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:22.871980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:22.872760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:22.873124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:22.876259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:22.911107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1850/3652] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:22.911906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:22.943991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f9a1-b71c-40c6-982b-a4813dead24f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:22.950268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:22.952251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:22.981643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:22.982286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:22.996346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.008360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.010520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.015664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.130157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599c58a-a7c6-4820-95cd-c97bb97a0cff admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.135669 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1803/3653] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:22 2022] POST /identity/v3/projects => generated 393 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:23.139523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:23.276302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d26169-9e30-4768-9c98-06c07f0e3782 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.291621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.292730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.293183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.293890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.294819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.297255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.327208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:23.347358 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1804/3654] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:23 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:23.349608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.350307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.350679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.351182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.351739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:23.353836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:23.367499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c03db-ac4a-4577-b3d7-d5b9488eb45d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.389060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.390643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.391543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.392801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.393700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.395946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.413881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:23.419889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1805/3655] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:23.427495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:23.466394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8eef-2981-40c8-97cb-b052d2bd0a8d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.472612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.474336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.474811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.475239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.476086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.478417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.533401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.535206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671b175-9e55-4061-89ef-2dea93825547 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.543848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1806/3656] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:23.545011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:23.553218 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1851/3657] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:03:23.561192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabbd79-5c79-4a82-981f-311d0e870477 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.567452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.568485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.568771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.569044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] PATH_INFO: `/v3/projects/7309c3d3d7e542b1a000490468edec55/users/07d598811ae8427c982bbea05de22daf/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.593166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7309c3d3d7e542b1a000490468edec55, user_id=07d598811ae8427c982bbea05de22daf, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:23.596871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:23.614783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1807/3658] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:23 2022] PUT /identity/v3/projects/7309c3d3d7e542b1a000490468edec55/users/07d598811ae8427c982bbea05de22daf/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:23.620257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679676fd-9a97-4eed-be3b-f46758c9e00c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.620539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679676fd-9a97-4eed-be3b-f46758c9e00c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.620808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679676fd-9a97-4eed-be3b-f46758c9e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:23.661818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-679676fd-9a97-4eed-be3b-f46758c9e00c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:23.663075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-679676fd-9a97-4eed-be3b-f46758c9e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:23.795630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1852/3659] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:23.810018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:23.815081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:23.967299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f1f52-a0fd-44db-b301-e9a3dd21b159 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:23.976897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:23.978242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:23.978576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:23.978853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:24.004601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:24.006086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed68f8-d03a-4055-9cd3-ffebc7058b23 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:24.043090 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1808/3660] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:26.257431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:26.258012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:26.258012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:26.338929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] MFA Rules not processed for user `5cb9430023d842b4a5d930e01f7340f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:26.348506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:26.528981 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1853/3661] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:26.541031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:26.542175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:26.542666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:26.566267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] MFA Rules not processed for user `5cb9430023d842b4a5d930e01f7340f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:26.567955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:26.577154 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1809/3662] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:26.651536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:26.748277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:26.788999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f524-d467-46b0-985e-233338ce7b05 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:26.800836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:26.801878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:26.802321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:26.802769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:26.817014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:26.819103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b925b5e-c0a1-4df3-83e6-06d165ccb6f1 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:26.850993 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1854/3663] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:27.053469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:27.057844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:27.100423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81a99-28db-4245-b524-834ea3d1e925 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:27.122417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:27.124599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:27.127125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:27.128653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:27.132066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:27.392233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:27.394086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:27.411946 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1810/3664] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:27.480801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf245a-55e2-4841-bab6-a67acde01871 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:27.485191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf245a-55e2-4841-bab6-a67acde01871 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:27.485686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf245a-55e2-4841-bab6-a67acde01871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:27.531331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d1cf245a-55e2-4841-bab6-a67acde01871 None None] MFA Rules not processed for user `7654d225081948cabad9d2cdbf03ebda`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:27.533276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1cf245a-55e2-4841-bab6-a67acde01871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:27.672602 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1855/3665] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:28.902710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:28.916441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0d023-e0f4-4722-ae0c-384d0eba525f service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:28.922880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:28.923942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:28.924393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:28.924814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:28.935677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:28.937569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa6bd-2823-4eb7-805c-3fd4df75d414 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:28.963025 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1811/3666] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:29.848871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b19a-a01c-42d0-8a57-db234fc4764e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:29.849323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b19a-a01c-42d0-8a57-db234fc4764e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:29.849412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593b19a-a01c-42d0-8a57-db234fc4764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:29.897011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1593b19a-a01c-42d0-8a57-db234fc4764e None None] MFA Rules not processed for user `5cb9430023d842b4a5d930e01f7340f3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:29.898158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1593b19a-a01c-42d0-8a57-db234fc4764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:29.937592 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1856/3667] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:34.936676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca957-9a85-4906-a736-7ddc0862bfcf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:34.937721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca957-9a85-4906-a736-7ddc0862bfcf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:34.938136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ca957-9a85-4906-a736-7ddc0862bfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:34.963744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1f8ca957-9a85-4906-a736-7ddc0862bfcf None None] MFA Rules not processed for user `7654d225081948cabad9d2cdbf03ebda`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:34.965292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f8ca957-9a85-4906-a736-7ddc0862bfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:34.974219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1812/3668] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:37.999728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:38.000265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:38.000406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:38.063648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] MFA Rules not processed for user `07d598811ae8427c982bbea05de22daf`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:38.064976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:38.218388 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1857/3669] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:38.227642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15b700-7715-41d5-a336-82b8df39af46 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:38.228531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15b700-7715-41d5-a336-82b8df39af46 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:38.228531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15b700-7715-41d5-a336-82b8df39af46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:38.256718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eb15b700-7715-41d5-a336-82b8df39af46 None None] MFA Rules not processed for user `07d598811ae8427c982bbea05de22daf`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:38.258154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb15b700-7715-41d5-a336-82b8df39af46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:38.270994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1813/3670] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:38.475508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:38.489279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8de123-f2ae-4e3e-a3f5-52eacac6cbe3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:38.497640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:38.498426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:38.499752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:38.500108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:38.511046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:38.512993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:38.523238 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1858/3671] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:44.050458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00522e-e39c-445a-bd6b-0890db56a9b2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:44.050931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00522e-e39c-445a-bd6b-0890db56a9b2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:44.051015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00522e-e39c-445a-bd6b-0890db56a9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:44.084659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ca00522e-e39c-445a-bd6b-0890db56a9b2 None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:44.085892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ca00522e-e39c-445a-bd6b-0890db56a9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:44.133725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:44.206354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1814/3672] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:46.838295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:46.840114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:46.916728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:46.917878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:46.918528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:46.923286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:46.954771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:46.955896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:46.965436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1815/3673] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:46.965721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9656-e175-4870-833f-baabc9fd04d9 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:46.974945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:46.975572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:46.975850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:46.976119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:46.978141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:46.990918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:46.993265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:47.005338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1859/3674] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:50.344130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:50.346333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:50.375213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a216e-dd17-4701-9ac3-9abc0ce5e1ec None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:50.392512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:50.394956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:50.399296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:50.401427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] PATH_INFO: `/v3/users/cea166f15a894cd7b1cee6c5a749679c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:50.404313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea166f15a894cd7b1cee6c5a749679c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:50.407138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:50.494252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea166f15a894cd7b1cee6c5a749679c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:50.494928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea166f15a894cd7b1cee6c5a749679c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:50.506275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea166f15a894cd7b1cee6c5a749679c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:50.517506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea166f15a894cd7b1cee6c5a749679c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:50.518399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-957f7dc2-9376-49a6-82f5-2d468bca33e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea166f15a894cd7b1cee6c5a749679c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:50.551099 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1816/3675] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:50 2022] DELETE /identity/v3/users/cea166f15a894cd7b1cee6c5a749679c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:50.845505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:50.865094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9d93-a41e-49c0-8cbf-f52dd80271b6 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:50.953094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:50.953901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:50.954291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:50.954721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] PATH_INFO: `/v3/projects/5e3dbfe14ac3469286aa595d46c28f56` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:50.956135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3dbfe14ac3469286aa595d46c28f56)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:50.958157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.014125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3dbfe14ac3469286aa595d46c28f56', 'request_id': 'req-d42e4585-93cc-4041-8be1-5159b1ac1309'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:51.030944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3dbfe14ac3469286aa595d46c28f56', 'request_id': 'req-d42e4585-93cc-4041-8be1-5159b1ac1309'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:51.042869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3dbfe14ac3469286aa595d46c28f56', 'request_id': 'req-d42e4585-93cc-4041-8be1-5159b1ac1309'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:51.062757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] The token cache is being invalidate because project 5e3dbfe14ac3469286aa595d46c28f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:51.063360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:51.064046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d42e4585-93cc-4041-8be1-5159b1ac1309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:51.066477 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1860/3676] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:50 2022] DELETE /identity/v3/projects/5e3dbfe14ac3469286aa595d46c28f56 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:51.071385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.071785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.072359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.094130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:51.096025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:51.193826 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1817/3677] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:51.196978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.197386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.197816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.220996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:51.223938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:51.238544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1861/3678] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:03:51.241180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.254308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62531b2f-c743-4f45-87af-13bb1241d379 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.259988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.260601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.261062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.261347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.262007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:51.262308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.263984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.297378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1818/3679] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:51.317620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.330150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db95b9-c0f7-470c-a8ee-fccd2450490a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.344917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.346018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.346378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.346856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.348092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.350086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.431033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1862/3680] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:51 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:51.434960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.456574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c97c7-f20e-4496-a855-98a1b28f05eb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.485016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:51.584137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.585694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.586244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.586926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.588485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.591536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.656545 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1819/3681] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:51 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:03:51.657302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.674889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2205d-82f2-4f96-b138-3afbe44fdfdd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.682868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.684152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.684451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.684889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.685686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.687714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.713864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.716803 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1863/3682] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:03:51.729650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248435-da5c-427b-944a-0df7b7c120ce admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.738519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.741341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.741898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.743890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.746517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.748419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b3479-6eb5-49c0-a9c1-0793da9f5e2c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.781132 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1820/3683] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:51.782375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:51.800995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43660a7c-99a9-4a08-b96b-d49b97f51b78 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:51.808321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:51.809129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.809929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.810287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] PATH_INFO: `/v3/projects/6b7f252278c74e2d84de081cd3d5a69e/users/dad0b98fdcf145a1960121573524940d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.830366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7f252278c74e2d84de081cd3d5a69e, user_id=dad0b98fdcf145a1960121573524940d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:51.833551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:51.839601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:51.858842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1864/3684] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:51 2022] PUT /identity/v3/projects/6b7f252278c74e2d84de081cd3d5a69e/users/dad0b98fdcf145a1960121573524940d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:51.860573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbab2-3dea-406a-bd28-91038f294cff None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:51.860933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbab2-3dea-406a-bd28-91038f294cff None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:51.861282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fbab2-3dea-406a-bd28-91038f294cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:51.883509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-863fbab2-3dea-406a-bd28-91038f294cff None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:51.884949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-863fbab2-3dea-406a-bd28-91038f294cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:52.000582 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1821/3685] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:03:52.032842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:52.035245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:52.195065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224973ed-72fa-4e25-a128-2abd16365ec4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:52.200132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:52.201120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:52.201745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:52.202455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:52.220946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:52.223753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:52.249410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1865/3686] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:54.244185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0627f8-0d92-414a-b0eb-85e48732b8db None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:54.244696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0627f8-0d92-414a-b0eb-85e48732b8db None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:54.244696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0627f8-0d92-414a-b0eb-85e48732b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:54.322140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0b0627f8-0d92-414a-b0eb-85e48732b8db None None] MFA Rules not processed for user `dad0b98fdcf145a1960121573524940d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:54.323963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0b0627f8-0d92-414a-b0eb-85e48732b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:54.471959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:54.473381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:54.490936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d022e23-33c8-45be-a340-1b0a6f549e9e service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:54.504622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:54.505278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:54.505579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:54.505905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] PATH_INFO: `/v3/users/7d8199c7aba04a9fb1212a2c7e1a52b3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:54.510228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8199c7aba04a9fb1212a2c7e1a52b3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:54.513729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:54.516744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1822/3687] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:54.530139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e7391-0094-4135-bd75-02f2c5d2f5c2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:54.530276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e7391-0094-4135-bd75-02f2c5d2f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:54.530482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e7391-0094-4135-bd75-02f2c5d2f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:54.572357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-424e7391-0094-4135-bd75-02f2c5d2f5c2 None None] MFA Rules not processed for user `dad0b98fdcf145a1960121573524940d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:54.589854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-424e7391-0094-4135-bd75-02f2c5d2f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:54.599062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1823/3688] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:54.612361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:54.612604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:54.613089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:54.634529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8199c7aba04a9fb1212a2c7e1a52b3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:54.635571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8199c7aba04a9fb1212a2c7e1a52b3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:54.648207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8199c7aba04a9fb1212a2c7e1a52b3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:54.651602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] MFA Rules not processed for user `dad0b98fdcf145a1960121573524940d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:54.654135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:54.671427 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1824/3689] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:54.673873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8199c7aba04a9fb1212a2c7e1a52b3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:54.675195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8199c7aba04a9fb1212a2c7e1a52b3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:54.707079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1866/3690] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:54 2022] DELETE /identity/v3/users/7d8199c7aba04a9fb1212a2c7e1a52b3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:03:54.728914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:54.730670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:54.755067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:54.860613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f97b00-07bb-441c-91d8-6be9c9fdda82 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:54.959452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:54.960334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:54.960869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:54.962551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:54.964735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:55.063807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.064956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f4ace-0663-40b0-82fc-a1620f52ee7a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.083637 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1825/3691] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:55.194209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.196636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:55.287162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4173d-35e5-42d7-9ff5-b01f0172ce98 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.292502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.293265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.293794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.294284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] PATH_INFO: `/v3/projects/dad3759161984bb896e1423af9a91d14` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.295685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad3759161984bb896e1423af9a91d14)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.298092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.312948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:55.369335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad3759161984bb896e1423af9a91d14', 'request_id': 'req-30740212-41cf-42dd-afbd-f724d07aaff8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:55.377534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad3759161984bb896e1423af9a91d14', 'request_id': 'req-30740212-41cf-42dd-afbd-f724d07aaff8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:55.385305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad3759161984bb896e1423af9a91d14', 'request_id': 'req-30740212-41cf-42dd-afbd-f724d07aaff8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:55.405644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] The token cache is being invalidate because project dad3759161984bb896e1423af9a91d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:55.406586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:55.407552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30740212-41cf-42dd-afbd-f724d07aaff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:55.410132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1867/3692] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:55 2022] DELETE /identity/v3/projects/dad3759161984bb896e1423af9a91d14 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:55.415603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.416146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.416146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.459645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:55.464884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:55.542035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:55.563056 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1826/3693] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:55.572387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.572899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.572899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.600367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:55.602164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:55.611851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1868/3694] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:55.616340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.629210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587882a-8cba-4f7a-abf5-94caa489fdd9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.635478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.636396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.636849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.637320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.638465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:55.638961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.641088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.662793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.673748 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1827/3695] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:55.674120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa8ca2-b13d-4aa4-9211-fbe67d812ea0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.682353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.683473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.683664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.683978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.684682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.686757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.743657 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1869/3696] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:55 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:55.762843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.774729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e361eb-3bfd-419d-b9a8-d89bcd307dc6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.872172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.872867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.873213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.873556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.874631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.876750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.904644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1828/3697] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:55 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:55.908855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.920280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651ec8c-bc36-49b4-bb31-20515e249d98 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.927856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.928553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.928898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.929248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.930568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.932256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.944197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1870/3698] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:03:55.948862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.959582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16f324-be5c-473b-acdd-0e95955152bc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:55.965237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:55.965949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:55.966289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:55.966629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:55.967251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:55.968876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0a12c-5cc5-4366-808e-160323fe602b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:55.981012 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1829/3699] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:55.985331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:55.996014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6930-5365-415d-8ccc-23ccd9831b2a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:56.000755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:56.001409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:56.001688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:56.001962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] PATH_INFO: `/v3/projects/a6220c46bd7a41ef93fd668a0483e0bd/users/31218f3711a640dbba5eabb3d70442b9/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:56.017855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6220c46bd7a41ef93fd668a0483e0bd, user_id=31218f3711a640dbba5eabb3d70442b9, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:56.020487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be43986-4fc4-4780-aff3-62d6d6b741b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:56.035165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1871/3700] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:55 2022] PUT /identity/v3/projects/a6220c46bd7a41ef93fd668a0483e0bd/users/31218f3711a640dbba5eabb3d70442b9/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:56.045982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:56.046269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:56.046572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:56.088778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] MFA Rules not processed for user `31218f3711a640dbba5eabb3d70442b9`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:56.090929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:56.170722 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1830/3701] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:56.786773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:56.787608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:56.788074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:56.833734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] MFA Rules not processed for user `dad0b98fdcf145a1960121573524940d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:56.837513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:56.940277 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1872/3702] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:57.072446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:57.073919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:57.177241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:57.254929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76296963-de27-4d0d-8dbe-ddcd2dbe6b6a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:57.270762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:57.271979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:57.272447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:57.272929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:57.284377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:57.285943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861197ce-6062-4bf1-b9a8-1bd4dcf2af5e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:57.307858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1831/3703] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:57.353866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:57.462731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:57.502647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f754b2f-9af0-4c25-ae7b-6a4e01e2b5a9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:57.510372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:57.511824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:57.512107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:57.512377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] PATH_INFO: `/v3/users/31218f3711a640dbba5eabb3d70442b9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:57.514162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31218f3711a640dbba5eabb3d70442b9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:57.517536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:57.599650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31218f3711a640dbba5eabb3d70442b9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:57.603551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31218f3711a640dbba5eabb3d70442b9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:57.625315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31218f3711a640dbba5eabb3d70442b9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:57.671789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31218f3711a640dbba5eabb3d70442b9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:57.672625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31218f3711a640dbba5eabb3d70442b9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:57.693878 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1873/3704] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:03:57 2022] DELETE /identity/v3/users/31218f3711a640dbba5eabb3d70442b9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:03:57.696171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a1d5e-2fb5-49dc-af8b-86ce06f0cef7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:57.696449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a1d5e-2fb5-49dc-af8b-86ce06f0cef7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:57.696749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a1d5e-2fb5-49dc-af8b-86ce06f0cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:57.746405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-601a1d5e-2fb5-49dc-af8b-86ce06f0cef7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:57.747002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-601a1d5e-2fb5-49dc-af8b-86ce06f0cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:57.870313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1832/3705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:03:57.884822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:57.886382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:57.970105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6625c-fdfd-4ec5-8caf-dfda7ee6c3b9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:57.974064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:57.974851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:57.975231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:57.975757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:57.986761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:57.988372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:57.999923 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1874/3706] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:03:58.290273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbdb76-6240-4bb6-b081-04292513549e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:58.291410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbdb76-6240-4bb6-b081-04292513549e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:58.291895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbdb76-6240-4bb6-b081-04292513549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:58.375567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-68bbdb76-6240-4bb6-b081-04292513549e None None] MFA Rules not processed for user `dad0b98fdcf145a1960121573524940d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:58.376801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-68bbdb76-6240-4bb6-b081-04292513549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:58.445520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:58.447291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:58.494035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ebc22-1e4d-4a2a-8c5a-640aeba62fb4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:58.507643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:58.508829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:58.510267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:58.511408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] PATH_INFO: `/v3/projects/a6220c46bd7a41ef93fd668a0483e0bd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:58.513698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6220c46bd7a41ef93fd668a0483e0bd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:58.516330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:58.551430 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1833/3707] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:58.586101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6220c46bd7a41ef93fd668a0483e0bd', 'request_id': 'req-31b20d4a-447d-4ea0-ad62-964ec0922fd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:58.593123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6220c46bd7a41ef93fd668a0483e0bd', 'request_id': 'req-31b20d4a-447d-4ea0-ad62-964ec0922fd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:58.594294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:58.607274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6220c46bd7a41ef93fd668a0483e0bd', 'request_id': 'req-31b20d4a-447d-4ea0-ad62-964ec0922fd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:58.623142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] The token cache is being invalidate because project a6220c46bd7a41ef93fd668a0483e0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:03:58.623432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:58.624499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31b20d4a-447d-4ea0-ad62-964ec0922fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:03:58.635473 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1875/3708] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:03:58 2022] DELETE /identity/v3/projects/a6220c46bd7a41ef93fd668a0483e0bd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:03:58.654407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:58.654407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:58.654407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:58.698954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:58.700635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:58.820155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:03:58.923216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1834/3709] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:58.932381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:58.933497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:58.934297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:58.986574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:58.987803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:58.998854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1876/3710] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:59.003850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.019479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78afa96-c602-4e31-abed-9fd125de01f5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.027033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.028107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.028372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.028641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.029349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:03:59.029645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.032375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.050573 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1835/3711] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:03:59.055396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.091652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc143d-535f-4845-8866-ce775c57c3c1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.099954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.100744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.116598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.117582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.120186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.130773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.183422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1877/3712] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:59.187437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.203285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018dd60-6fab-4e3b-983d-1311dc0ca0f5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.293870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.294624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.295101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.295311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.296028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.298298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.330591 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1836/3713] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:03:59 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:03:59.333332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.347017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4404-f1c8-4dce-846a-069e9bc92bb6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.354114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.355291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.355865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.356148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.356801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.358989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.374346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1878/3714] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:59.378670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.401918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9b479-8008-4f97-8f8c-cd467d5e144d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.417855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.418723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.419007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.419281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.419925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.421516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fa806-adef-44e2-bcbf-faa9170ac11e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.457734 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1837/3715] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:03:59.462541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.483690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c253756-f37b-4e27-9eba-218fa2166fcc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.489305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.490230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.490793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.491190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] PATH_INFO: `/v3/projects/2b0ea77e1f9848e3b286923e271c1c91/users/4f1841a68cb1458588cf949d1eb8a00c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.514245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0ea77e1f9848e3b286923e271c1c91, user_id=4f1841a68cb1458588cf949d1eb8a00c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.517529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.535208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1879/3716] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:03:59 2022] PUT /identity/v3/projects/2b0ea77e1f9848e3b286923e271c1c91/users/4f1841a68cb1458588cf949d1eb8a00c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:03:59.540175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05fa5-6f35-49d2-b719-4e724d977a18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.540545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05fa5-6f35-49d2-b719-4e724d977a18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.540891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05fa5-6f35-49d2-b719-4e724d977a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.606519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-09a05fa5-6f35-49d2-b719-4e724d977a18 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:03:59.608177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09a05fa5-6f35-49d2-b719-4e724d977a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:59.730016 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1838/3717] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:03:59.739885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:03:59.742810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:03:59.828288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b0a42-5f86-4f14-95bc-5485ccef42b9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:03:59.833764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:03:59.835450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:03:59.835939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:03:59.851185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:03:59.865493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:03:59.867813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ca802-46ba-4cd1-8f86-5e9b20aeb11c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:03:59.902743 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1880/3718] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:04:00.771716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:00.772178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:00.772276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:00.816730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:00.840848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] MFA Rules not processed for user `4f1841a68cb1458588cf949d1eb8a00c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:00.842348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:00.977687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:00.978788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:00.979287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:00.986392 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1839/3719] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:01.022964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] MFA Rules not processed for user `4f1841a68cb1458588cf949d1eb8a00c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:01.024856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:01.043163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1881/3720] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:01.298388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:01.300885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:01.428541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adc9e9-026f-45a6-82bd-9ecd59e80bd3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:01.437474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:01.438230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:01.438610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:01.438963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:01.452004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:01.455539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:01.469503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1840/3721] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:03.888198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:03.901983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbba9a-8e96-4303-a93f-d5d3d0335ff0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:03.907481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:03.908654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:03.909106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:03.909623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] PATH_INFO: `/v3/users/4f1841a68cb1458588cf949d1eb8a00c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:03.911100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1841a68cb1458588cf949d1eb8a00c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:03.913353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:04.004775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1841a68cb1458588cf949d1eb8a00c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:04.005487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1841a68cb1458588cf949d1eb8a00c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:04.016206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1841a68cb1458588cf949d1eb8a00c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:04.028664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:04.051938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1841a68cb1458588cf949d1eb8a00c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:04.052834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d85c3fe0-83d2-44a2-9006-6ca4344a5eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1841a68cb1458588cf949d1eb8a00c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:04.078359 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1882/3722] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:03 2022] DELETE /identity/v3/users/4f1841a68cb1458588cf949d1eb8a00c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:04.730962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:04.744348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12ca4-db79-4802-acbc-28460a676d85 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:04.933742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:04.935409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:04.935698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:04.936159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] PATH_INFO: `/v3/projects/2b0ea77e1f9848e3b286923e271c1c91` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:04.937426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b0ea77e1f9848e3b286923e271c1c91)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:04.950049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:04.955167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:05.003120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b0ea77e1f9848e3b286923e271c1c91', 'request_id': 'req-21a0849f-3a63-415e-886b-30d32241d27a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:05.012206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b0ea77e1f9848e3b286923e271c1c91', 'request_id': 'req-21a0849f-3a63-415e-886b-30d32241d27a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:05.024357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b0ea77e1f9848e3b286923e271c1c91', 'request_id': 'req-21a0849f-3a63-415e-886b-30d32241d27a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:05.040390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] The token cache is being invalidate because project 2b0ea77e1f9848e3b286923e271c1c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:05.040941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:05.041859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-21a0849f-3a63-415e-886b-30d32241d27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:05.043803 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1841/3723] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:04 2022] DELETE /identity/v3/projects/2b0ea77e1f9848e3b286923e271c1c91 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:05.048658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.049161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.049248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.096951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:05.098452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:05.191216 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1883/3724] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:05.196214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.196503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.196782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.234434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:05.235851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:05.253852 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1842/3725] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:05.255051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.266294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73023a89-dc48-449f-8610-d3d3acff50cb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.271501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.272363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.272709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.273813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.274612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:05.275018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.276749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.293915 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1884/3726] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:05.298635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.308126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8c281-f2d5-437f-95bd-b9ae689be08a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.313119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.313815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.314159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.314523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.315249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.317232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.375667 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1843/3727] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/projects => generated 391 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:05.377455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.392413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67b998-ede9-4fd3-bf5b-248d51841a37 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.478535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.478535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.478535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.478535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.479013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.479958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.524329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.528224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1885/3728] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:05.539156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ee142-7c6e-44f5-8171-87f135efacc7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.545110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.546363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.547142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.547142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.547915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.549543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.577479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.578588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1844/3729] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:05.597576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9d7f7-7428-43d6-b799-d638815b1e75 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.610888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.611764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.612194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.612543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.613311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.615050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221a89-c861-4447-94a9-84b67767367b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.634551 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1886/3730] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:05.645629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:05.678097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6485f3a-b5f9-4f1a-99ae-0d764c078f4c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:05.694742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:05.697014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.697384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.697833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] PATH_INFO: `/v3/projects/8eb4e8d763894f74868a69aac2154172/users/4ebbd6dc51c14970a728b7b4fec35b55/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.726871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb4e8d763894f74868a69aac2154172, user_id=4ebbd6dc51c14970a728b7b4fec35b55, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:05.729736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:05.755178 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1845/3731] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:05 2022] PUT /identity/v3/projects/8eb4e8d763894f74868a69aac2154172/users/4ebbd6dc51c14970a728b7b4fec35b55/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:05.767460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:05.768116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:05.768610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:05.800290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] MFA Rules not processed for user `4ebbd6dc51c14970a728b7b4fec35b55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:05.801519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:05.828262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:05.918899 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1887/3732] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:05.923493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.022714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f0047-4b9e-4576-a3eb-c514becb7fb7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.027681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.028306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.028584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.028858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.029511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.031589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.064202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1846/3733] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:05 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:06.068485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.082287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c1fb5-649c-476f-b8b7-b28803ee3c5f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.153721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.154516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.154840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.155762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.157678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.160543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.191066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1888/3734] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:06 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:06.192279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.203441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd7e5-b2af-45c5-ba7b-e5ca2c69781b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.209611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.210317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.211133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.211962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.218786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.220585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.237023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1847/3735] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:04:06.238306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.254401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9d895-64bb-4c47-800b-c8296ee9f8da admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.260299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.261145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.261440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.261714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] PATH_INFO: `/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5/users/ebc79ddabeb34c5b8bd6579250093745/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.279859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7b23feefa143999e89f42a8d3dcbc5, user_id=ebc79ddabeb34c5b8bd6579250093745, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.283369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.297953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1889/3736] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:06 2022] PUT /identity/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5/users/ebc79ddabeb34c5b8bd6579250093745/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:06.301955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.308216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:06.313839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faed9-f996-4d96-8378-137e4640c179 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.383552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.384379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.384738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.386198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.387652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.390223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a6095-f3ac-4a4f-bf2c-749ac881a845 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.404207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1848/3737] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:06.407759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.418846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff65f8-0a83-4527-afe2-b04208731520 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.423785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.424541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.424899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.425249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] PATH_INFO: `/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5/users/ebc79ddabeb34c5b8bd6579250093745/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.427339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7b23feefa143999e89f42a8d3dcbc5, user_id=ebc79ddabeb34c5b8bd6579250093745, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.431490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.454442 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1890/3738] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:06 2022] PUT /identity/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5/users/ebc79ddabeb34c5b8bd6579250093745/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:06.455718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.456013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.456289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.493153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] MFA Rules not processed for user `ebc79ddabeb34c5b8bd6579250093745`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:06.494439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:06.566938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1849/3739] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:06.572365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.583562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7eb86e-b6ed-4e0b-9175-5ed94349b0c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.589371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.590060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.590348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.590667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.591514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.592524 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.592524 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:06.592524 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:06.593420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.614598 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1891/3740] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:04:06 2022] POST /identity/v3/domains => generated 275 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:06.618226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.627886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb8adc-712d-4d12-895c-53818cd8b959 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.693080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.693743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.694023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.694304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.695013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=276cc519839e42cda00c4e79605664dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.696324 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.696324 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:06.696324 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:06.697273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.730596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:06.742750 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1850/3741] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:06 2022] PUT /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config => generated 138 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:06.746411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.757058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f721c-898d-4aba-928a-1ebfe275cf54 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.762966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.763698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.763972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.764241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.765374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=276cc519839e42cda00c4e79605664dc, group=identity)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.766383 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.766383 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:06.766383 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:06.767331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.775231 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1892/3742] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config/identity => generated 45 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:06.779686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.792010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffded5c-84f2-4246-ba0e-187ab2cac720 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.799331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.800104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.800489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.800871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config/identity/driver` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.802227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=276cc519839e42cda00c4e79605664dc, group=identity, option=driver)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.804270 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.804270 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:06.804270 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:06.805934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.827168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1851/3743] 10.222.0.52 () {58 vars in 1296 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config/identity/driver => generated 31 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:06.830984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.842247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a11e9-e146-46e9-975f-e0d58e1db3b9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.862446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.863251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.863534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.863809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.865024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=276cc519839e42cda00c4e79605664dc, group=ldap)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.867335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.874890 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1893/3744] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap => generated 106 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:06.879234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.889680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a96e9b-0599-4528-91fb-cc7d11a5a4f5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.894355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.895069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.895351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.895625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap/url` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.896784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=276cc519839e42cda00c4e79605664dc, group=ldap, option=url)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.898577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.903373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1852/3745] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap/url => generated 46 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:06.907534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.918839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab3294-06c5-4e60-b1e4-6ce936140b80 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.926789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.927519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.927808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.928085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap/user_tree_dn` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.930124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=276cc519839e42cda00c4e79605664dc, group=ldap, option=user_tree_dn)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.931771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.938817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1894/3746] 10.222.0.52 () {58 vars in 1302 bytes} [Sat Sep 24 21:04:06 2022] GET /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config/ldap/user_tree_dn => generated 63 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:06.942871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.952813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059d48-1ae1-4397-bbdf-6f1dc93681e6 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.957503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.958118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.958409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.958845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.959423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=276cc519839e42cda00c4e79605664dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.960973 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.960973 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:06.960973 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:06.962018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:06.971645 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1853/3747] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:06 2022] DELETE /identity/v3/domains/276cc519839e42cda00c4e79605664dc/config => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:06.975944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:06.988607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a720be-b381-46f9-9741-6cabfc354ff0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:06.993743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:06.994707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:06.995154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:06.995520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:06.996777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=276cc519839e42cda00c4e79605664dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:06.998681 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:06.998681 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:06.998681 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:06.999642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.036579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.038332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.066405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:07.093798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.094570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.098289 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1895/3748] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:06 2022] PATCH /identity/v3/domains/276cc519839e42cda00c4e79605664dc => generated 276 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:07.102623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.104524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:07.119105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf20bd4-36c8-4391-aeb3-590677a3de97 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.215344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.216355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.216790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.217290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.231764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=276cc519839e42cda00c4e79605664dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.233639 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:07.233639 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:07.233639 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:07.235481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.265525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.293841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '276cc519839e42cda00c4e79605664dc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.358263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.365411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.373956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.387183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] The token cache is being invalidate because project 276cc519839e42cda00c4e79605664dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:07.387493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.388488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.407808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.408848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.412853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '276cc519839e42cda00c4e79605664dc', 'request_id': 'req-63913ba9-524e-4d37-bbbc-111b0f546301'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.420522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1854/3749] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:07 2022] DELETE /identity/v3/domains/276cc519839e42cda00c4e79605664dc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:07.428332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.444283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec106ef1-2001-41dd-b27e-22235c6a7fa0 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.518359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.519223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.519649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.519923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.520720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.522417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.546091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1896/3750] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:04:07 2022] POST /identity/v3/domains => generated 276 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:07.547324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.548754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:07.559976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:07.564475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913ba9-524e-4d37-bbbc-111b0f546301 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.640667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.644850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.644850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.644850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.644850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.646176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.690823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1855/3751] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:07 2022] PUT /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 138 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:07.692024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.703013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0546b-f60e-4802-bb34-1520807cbd18 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.707650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.708285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.708565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.708891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.709523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.711184 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:07.711184 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:07.711184 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:07.712029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.763091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1897/3752] 10.222.0.52 () {60 vars in 1271 bytes} [Sat Sep 24 21:04:07 2022] PATCH /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 141 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:07.770984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.783814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350870c0-a1a3-4836-8ab1-103774d705d7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.789449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.790104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.790382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.790687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.791890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.793484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.798241 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1856/3753] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:07 2022] GET /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 141 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:07.802714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.813473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58841e-5422-46ad-a88b-ed770d9027be tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.818538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.819178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.819458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.819731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.820347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.821353 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:07.821353 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:07.821353 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:07.822228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.832962 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1898/3754] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:07 2022] DELETE /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:07.838901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.852195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9d15f-f249-4d28-b61a-558b9bde6909 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.857071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.857764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.858120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.858577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.859781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.861777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find any options in domain configuration for domain f87c93e74f1044e6b358297b3604955c.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain f87c93e74f1044e6b358297b3604955c. Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 24 21:04:07.871184 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain f87c93e74f1044e6b358297b3604955c. Sep 24 21:04:07.872369 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:07.879503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1857/3755] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:07 2022] GET /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 151 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:04:07.879676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.888148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e6b7-b82b-4fa8-86ed-f1aa5843080a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.893004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.893642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.894010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.894357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c/config` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.895037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.896834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.903911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1899/3756] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:07 2022] DELETE /identity/v3/domains/f87c93e74f1044e6b358297b3604955c/config => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:07.908213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.919530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d412062-7575-4f4b-a5ec-4dece19de890 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:07.926187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:07.926925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:07.927269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:07.927608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:07.928887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:07.930337 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:07.930337 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:07.930337 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:07.931451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:07.953168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.954001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.982949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.983572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:07.989011 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1858/3757] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:07 2022] PATCH /identity/v3/domains/f87c93e74f1044e6b358297b3604955c => generated 277 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:07.990332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:07.992637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:08.005729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66aa95-24a3-4de3-aaf1-235fa282160c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.066965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.067600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.068008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.068292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.079997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f87c93e74f1044e6b358297b3604955c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.081015 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:08.081015 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:08.081015 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:08.081930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.122759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.122759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f87c93e74f1044e6b358297b3604955c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.127459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:08.192545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.204299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.211564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.227562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] The token cache is being invalidate because project f87c93e74f1044e6b358297b3604955c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:08.228198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.228908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.239409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.245018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.246400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f87c93e74f1044e6b358297b3604955c', 'request_id': 'req-669a59f7-6b65-4d1c-91c1-be82dc1e613b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:08.270081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1900/3758] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:07 2022] DELETE /identity/v3/domains/f87c93e74f1044e6b358297b3604955c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:08.275280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.286144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed12cb0-bfcb-463a-a0d2-2e4e5817fe37 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.354473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.355270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.355547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.355959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.356640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.357805 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:08.357805 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:08.357805 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:08.359725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.381423 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1859/3759] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:04:08 2022] POST /identity/v3/domains => generated 277 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:08.383835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.385699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:08.401443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a59f7-6b65-4d1c-91c1-be82dc1e613b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.470688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.471545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.471982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.472254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.473567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=624428207bc646419679aac3b43029ad)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.476049 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:08.476049 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:08.476049 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:08.479133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.530203 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1901/3760] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:08 2022] PUT /identity/v3/domains/624428207bc646419679aac3b43029ad/config => generated 138 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:08.531729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.549016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5d09b-3bbc-4dfa-8afb-630c6f96b662 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.562249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.563325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.564626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.565908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.567189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.569744 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:08.569744 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:08.569744 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:08.572262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.584981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:08.626911 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1860/3761] 10.222.0.52 () {60 vars in 1297 bytes} [Sat Sep 24 21:04:08 2022] PATCH /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity => generated 175 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:08.628192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.642698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6f32b-37e0-4b6c-bb9f-e7a526885246 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.668617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.670162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.672530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.673239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.676131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity, option=driver)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.679423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.715514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1902/3762] 10.222.0.52 () {60 vars in 1318 bytes} [Sat Sep 24 21:04:08 2022] PATCH /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver => generated 176 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:08.719704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.733176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db52af-f8e3-484e-b450-7a07c4ee0a93 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.739403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.740348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.740827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.741293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.743286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity, option=driver)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.745639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.789668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1861/3763] 10.222.0.52 () {58 vars in 1299 bytes} [Sat Sep 24 21:04:08 2022] DELETE /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:08.794544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.808194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36483839-41d0-4b3c-8b66-b5b39be70317 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.821187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.821879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.822169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.822447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.825911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity, option=driver)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.827625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find option driver in group identity in domain configuration for domain 624428207bc646419679aac3b43029ad.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain 624428207bc646419679aac3b43029ad. Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:08.848485 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain 624428207bc646419679aac3b43029ad. Sep 24 21:04:08.849564 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:08.854610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:08.855818 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1903/3764] 10.222.0.52 () {58 vars in 1296 bytes} [Sat Sep 24 21:04:08 2022] GET /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity/driver => generated 171 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:04:08.877067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0565d5-4952-4f33-887f-35307515d224 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:08.889032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:08.889637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:08.889929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:08.890209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:08.893936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:08.896227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:08.947855 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1862/3765] 10.222.0.52 () {58 vars in 1278 bytes} [Sat Sep 24 21:04:08 2022] DELETE /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:08.948828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:09.215876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec3be-1abc-4ae5-9017-99901091af0f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:09.221364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:09.222217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:09.222524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:09.223296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:09.224755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=624428207bc646419679aac3b43029ad, group=identity)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:09.227823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:09.230154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find group identity in domain configuration for domain 624428207bc646419679aac3b43029ad.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain 624428207bc646419679aac3b43029ad. Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:09.246432 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain 624428207bc646419679aac3b43029ad. Sep 24 21:04:09.247579 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:09.252627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1904/3766] 10.222.0.52 () {58 vars in 1275 bytes} [Sat Sep 24 21:04:08 2022] GET /identity/v3/domains/624428207bc646419679aac3b43029ad/config/identity => generated 154 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:04:09.253280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:09.283213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5cbb-cbe8-479a-8a8d-88df30a45603 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:09.293790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:09.294462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:09.294771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:09.295053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad/config` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:09.296325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=624428207bc646419679aac3b43029ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:09.297547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:09.313923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1863/3767] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:09 2022] DELETE /identity/v3/domains/624428207bc646419679aac3b43029ad/config => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:09.319518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:09.339453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669fb12-394e-410c-8108-3050698dda7c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:09.347274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:09.347274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:09.348486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:09.348783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:09.350976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=624428207bc646419679aac3b43029ad)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:09.353861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:09.387189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.388154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.422969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.425726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.428474 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1905/3768] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:09 2022] PATCH /identity/v3/domains/624428207bc646419679aac3b43029ad => generated 278 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:09.434672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:09.436987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:09.453895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7920a9-bd00-4640-892f-078aa598f1ac tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:09.539311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:09.539974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:09.540332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:09.540672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:09.556725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=624428207bc646419679aac3b43029ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:09.559490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:09.599005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.612379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '624428207bc646419679aac3b43029ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.738922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.751394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:09.754277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.769114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.800693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] The token cache is being invalidate because project 624428207bc646419679aac3b43029ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:09.801408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.803080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.839916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.842862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.853927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '624428207bc646419679aac3b43029ad', 'request_id': 'req-69d1076f-868e-4710-a28c-c862aa734130'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:09.869413 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1864/3769] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:09 2022] DELETE /identity/v3/domains/624428207bc646419679aac3b43029ad => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:09.874401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:09.885779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721ba4-fc3c-4e03-9804-e223f5ae1eb8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.017340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.017610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.018032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.018307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.018918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.019953 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:10.019953 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:04:10.019953 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:04:10.021011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.041159 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1906/3770] 10.222.0.52 () {58 vars in 1173 bytes} [Sat Sep 24 21:04:09 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:10.042010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.043831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:10.079599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1076f-868e-4710-a28c-c862aa734130 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.086625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.087296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.087543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.087821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.088525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.098861 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:04:10.098861 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:04:10.098861 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:04:10.099799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.105991 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1865/3771] 10.222.0.52 () {58 vars in 1200 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.109930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.126421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2ac65-7e19-4937-a424-a0774fb1374b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.149350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.152705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.153467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.155283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.157157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.162169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.167438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1907/3772] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.171899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.197175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a090eaa-db4b-4cf6-b41c-5daf9c0a7b15 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.218409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.221546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.222119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.222583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.223989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.225537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.228055 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1866/3773] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.231669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.267337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607e5a1-4a17-405a-b9af-76f962fa4191 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.275295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.276345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.276864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.278086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.279325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.282380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.284919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1908/3774] 10.222.0.52 () {58 vars in 1188 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:10.291009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.305536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd24269-5d01-4d20-9c02-37b47fb3aeb3 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.335270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.337090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.338369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.339284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.340920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.345416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.350307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1867/3775] 10.222.0.52 () {58 vars in 1200 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.355174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.373494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717edc6-0637-4bc0-b864-8eaca04f2cd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.388528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.389189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.389469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.389740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.390326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.391954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.396817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1909/3776] 10.222.0.52 () {58 vars in 1203 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.401748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.413238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09feb9f-526f-453c-abc0-8a5b34de184d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.441488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.441991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.442276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.442811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.443513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.445556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.448065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1868/3777] 10.222.0.52 () {58 vars in 1209 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.452021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.463187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5f2f7-698e-4913-a38a-292ae085ccba tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.468652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.469335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.469615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.469889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.470486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.472471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.474793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1910/3778] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.479782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.491190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1775c8-0592-43c1-838c-7c4c06ffeb21 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.507181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.510245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.512107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.513174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.515357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.517141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.518552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1869/3779] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.523208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.539697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6e5e9-d1f2-45a3-8952-ee9685a8dbd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.549738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.550411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.550730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.551006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.552160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.561896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.568548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1911/3780] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.570135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.582641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8387f80-10f3-4f0a-a53c-f1a503adbfa4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.588517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.589201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.589469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.589750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.590348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.592201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.594266 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1870/3781] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.599982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.615838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6835-d43d-4822-a153-89f5d6f5d267 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.623359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.624080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.624359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.624630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.625216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.627151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.633298 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1912/3782] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.634455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.646680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8296e0-6ca2-436c-a3db-62846dd6eb8d tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.654062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.655123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.655588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.655875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.656658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.658716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.663074 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1871/3783] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.665797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.678710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fe0a8-28aa-4b5f-8504-9214b52a0ef1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.686077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.687068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.687580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.691429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.692191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.694614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.696572 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1913/3784] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.700908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.713311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebdeaa-c857-42ce-8846-aa31718adefa tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.718274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.719093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.719479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.719879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.720567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.722316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.723716 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1872/3785] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.728691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.750550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fefd0-6b92-43dc-b5c9-45a9c51a0d5c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.756381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.756798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.757515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.758041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.758733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.760319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.761567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1914/3786] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.765584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.779521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37904668-0440-416a-8ac6-70adc5be3f86 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.784984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.785711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.786052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.786590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.787395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.790393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.791801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1873/3787] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 24 21:04:10.796002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.806230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de66a-8b13-42b3-8710-cd0628ca1e09 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.810908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.811547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.811878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.812096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.812759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.814296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.815511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1915/3788] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 24 21:04:10.819313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.829878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ddfc-92d0-4d5a-81d0-be40cb3bca94 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.837902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.838598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.838882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.839158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.839744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.841641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.842746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1874/3789] 10.222.0.52 () {58 vars in 1269 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.846692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.859610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2e6a-eb05-4d35-be27-738799e063e1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.864724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.865431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.865778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.866114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.867291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.869069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.870468 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1916/3790] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.874702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.885708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398639a4-7fa4-4e3b-a84b-02c4ca3b4424 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.890866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.891508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.891788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.892062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.892643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.894244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.895465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1875/3791] 10.222.0.52 () {58 vars in 1257 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.900413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.911207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf07587-2986-4c13-af36-6b734cfe898a tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.920284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.921047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.921475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.921819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.922623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.924488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.930411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:10.930676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1917/3792] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 24 21:04:10.953533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d2c1-e286-4f77-8f14-b34e58056dcb tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:10.981897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:10.984940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:10.986917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:10.987223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:10.991296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:10.993410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:10.998269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1876/3793] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:10.999677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.028743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc555ba2-f723-40c1-934d-3194414c78fe tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.045341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.047101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.047445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.047835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.048697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.050574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.053356 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1918/3794] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:10 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.058711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.085137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e5bbe-23ad-426e-b066-b7938f81110e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.091382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.092934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.093378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.093808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.094628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.096619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.103385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.114919 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1877/3795] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.120150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8cd96-3dbe-41d8-81dc-f88a2af07620 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.126358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.127322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.128310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.128772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.129660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.131493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.137796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.159942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1919/3796] 10.222.0.52 () {58 vars in 1290 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.165344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd618d6-d763-492d-860e-4babf7f03f90 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.176952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.178618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.179715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.180521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.183377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.185866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.192186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.203489 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1878/3797] 10.222.0.52 () {58 vars in 1257 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.206337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84728de1-c650-412a-ae47-affe481290e7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.213990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.215077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.215415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.215810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.216706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.218839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.220088 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1920/3798] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 24 21:04:11.224671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.238368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd40bea-e357-4be3-a638-6a8035087b0e tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.247521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.248558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.249045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.249607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.250451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.252564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.258776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1879/3799] 10.222.0.52 () {58 vars in 1308 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.259639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.278929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6aac8-8b25-4522-ae2c-c7ad3d1f4157 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.284321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.285067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.286244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.286648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.287429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.289840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.291563 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1921/3800] 10.222.0.52 () {58 vars in 1290 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.295526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.306896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8c49-4e51-45e3-bce4-a5290142ecd1 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.312294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.313071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.313426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.313899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.314445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.316680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.330526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1880/3801] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.334285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.347534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330484-6754-4b53-95b7-b5fb988a6c62 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.355962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.356635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.356914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.357184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.357772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.359576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.361725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1922/3802] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.365600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.381757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb6bce-5ee4-40a4-96ba-8963925883a7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.390383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.391367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.391845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.392340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.399052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.401040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.402850 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1881/3803] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.410108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.421580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb275-5f21-489f-ad01-d688407c556f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.426810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.428142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.430275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.430990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.431810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.435120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.438236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1923/3804] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) Sep 24 21:04:11.442859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.455840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a8e48-2065-4e2c-b444-2feb45fce889 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.474776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.477119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.477119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.477409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.477957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.485786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.489990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1882/3805] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.493287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.514048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960449c-78f2-4f20-abbf-e93ea132ca00 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.526154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.526843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.527122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.531518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.532222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.534102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.539536 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1924/3806] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.544892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.557411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68d08c-656a-458a-8485-a6d82d4829b5 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.563463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.564129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.564413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.564692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.565593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.567961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.573110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1883/3807] 10.222.0.52 () {58 vars in 1257 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.574797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.587284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6819ec8-72ae-4c83-8cde-791325fbac72 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.592905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.593800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.594290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.594949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.595654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.597623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.602620 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1925/3808] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.603910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.615606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a36f8b-3cf9-4aeb-abf8-a3dd7ced1e68 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.622960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.625048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.625567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.626057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.633783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.635608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.640940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1884/3809] 10.222.0.52 () {58 vars in 1257 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.645648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.668242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35144eff-24ed-4dce-b1f7-d750fe97b2b7 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.677430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.678101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.678385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.678687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.685487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.690249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.694924 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1926/3810] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.699261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.715937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868bc3c-47b1-4969-800f-e6d77a2ae5d4 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.721897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.722201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.722408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.722752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.723332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.724921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.726897 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1885/3811] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.731239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.742659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e4446-632a-4012-9fdc-2e406a9f47a8 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.749725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.750364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.750678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.750949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.751531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.753215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.754269 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1927/3812] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.758814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.769314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a27aa-07a9-4aa0-a054-e10808ddcc6f tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.777146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.777146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.777146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.777504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.777504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.778964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.780170 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1886/3813] 10.222.0.52 () {58 vars in 1212 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.784540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.795549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaa821-4403-48cd-ba43-461bcfecda36 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.801189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.802026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.803598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.803702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.807678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.807678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.807866 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1928/3814] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.813268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.827492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ba5b2-9196-40ca-8421-615e652affae tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.833327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.833972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.834246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.834711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.836045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.841106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.842215 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1887/3815] 10.222.0.52 () {58 vars in 1215 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.846108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.858773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c721c-35e8-4033-9460-b51116d7fbdc tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.875621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.876463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.877039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.877550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.878833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.880700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.882935 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1929/3816] 10.222.0.52 () {58 vars in 1218 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.887798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.903580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91a9b6-5e9f-4d14-86dc-cb3a1d5bb648 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.916342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.917015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.917336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.917818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.918593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.920768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.922382 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1888/3817] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.926700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.938476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01645144-ab6c-49c4-b7f9-bf8a4d246e6c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.944289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.946268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.946742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.947285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.948211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.950094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.951533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1930/3818] 10.222.0.52 () {58 vars in 1239 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.955663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:11.971682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b90217-85a1-4d30-8f84-1e0043087893 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:11.978735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:11.979576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:11.980147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:11.980577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:11.981169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:11.982815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:11.984547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1889/3819] 10.222.0.52 () {58 vars in 1260 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:11.988704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.005673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d616cb7-410c-4d99-914b-cdc315522be9 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.011738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.016288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.016288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.016288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.016288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.016682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.016922 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1931/3820] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:04:11 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:12.021430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.035404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba22e4-796a-47a6-8da9-d0e328897d3c tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.046473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.047174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.047459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.047738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.048329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.049966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.051538 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1890/3821] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:04:12 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:12.056257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.070238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1509c-8980-4666-890f-3efd70fc039b tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.078034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.078843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.079505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.079505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.079900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.087990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1bca5-ee55-4339-acb4-c942ebec5f73 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.089196 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1932/3822] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:12 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:12.093404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.108036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947c9ec-ad10-4299-a180-4e954a3928da tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.119264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.119691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.119810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.122903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.123564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.123845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.124122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.124705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.128429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.130409 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1891/3823] 10.222.0.52 () {58 vars in 1278 bytes} [Sat Sep 24 21:04:12 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:04:12.138921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.143561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:12.155812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:12.157053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:12.160323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06f5f9-e503-4d00-8fdc-34a6fdb3f799 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.168699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.169363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.169643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.170469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/624428207bc646419679aac3b43029ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.185849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find domain: 624428207bc646419679aac3b43029ad.: keystone.exception.DomainNotFound: Could not find domain: 624428207bc646419679aac3b43029ad. Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 624428207bc646419679aac3b43029ad. Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:12.190808 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:12.191961 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:04:12.192997 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 624428207bc646419679aac3b43029ad. Sep 24 21:04:12.194090 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.194090 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1892/3824] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:12 2022] PATCH /identity/v3/domains/624428207bc646419679aac3b43029ad => generated 112 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:04:12.196330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.209595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da6a8f-58b3-425e-9aa4-58cb16e74c63 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.220503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.221436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.221436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.221636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/f87c93e74f1044e6b358297b3604955c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.232659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:04:12.238548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1933/3825] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find domain: f87c93e74f1044e6b358297b3604955c.: keystone.exception.DomainNotFound: Could not find domain: f87c93e74f1044e6b358297b3604955c. Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f87c93e74f1044e6b358297b3604955c. Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:12.246925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:12.248051 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:04:12.249082 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f87c93e74f1044e6b358297b3604955c. Sep 24 21:04:12.250154 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:12.252305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1893/3826] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:12 2022] PATCH /identity/v3/domains/f87c93e74f1044e6b358297b3604955c => generated 112 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:04:12.258319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.275154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b28333-9dc2-45a0-98fa-8289b82871eb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.285124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc79ddabeb34c5b8bd6579250093745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee7b23feefa143999e89f42a8d3dcbc5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.285931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.286264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.289507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] PATH_INFO: `/v3/domains/276cc519839e42cda00c4e79605664dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.298594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-1cf06930-02a2-441c-bd03-4d611338da16 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Could not find domain: 276cc519839e42cda00c4e79605664dc.: keystone.exception.DomainNotFound: Could not find domain: 276cc519839e42cda00c4e79605664dc. Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 276cc519839e42cda00c4e79605664dc. Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:12.306006 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:12.307075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:04:12.308066 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 276cc519839e42cda00c4e79605664dc. Sep 24 21:04:12.309095 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:12.314311 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1934/3827] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:04:12 2022] PATCH /identity/v3/domains/276cc519839e42cda00c4e79605664dc => generated 112 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:04:12.315271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.317632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:12.389543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:12.463392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63938991-ea2c-4fd3-80ed-5cae6f179297 tempest-DomainConfigurationTestJSON-449157411 tempest-DomainConfigurationTestJSON-449157411-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.483776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.484329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.484626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.484887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] PATH_INFO: `/v3/users/4ebbd6dc51c14970a728b7b4fec35b55` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.486264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebbd6dc51c14970a728b7b4fec35b55)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.488554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.564022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebbd6dc51c14970a728b7b4fec35b55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:12.564810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebbd6dc51c14970a728b7b4fec35b55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:12.575291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebbd6dc51c14970a728b7b4fec35b55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:12.589370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebbd6dc51c14970a728b7b4fec35b55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:12.590478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebbd6dc51c14970a728b7b4fec35b55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:12.612056 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1894/3828] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:12 2022] DELETE /identity/v3/users/4ebbd6dc51c14970a728b7b4fec35b55 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:12.620164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.620458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.620733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.640399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:12.641949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:12.722010 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1935/3829] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:12.732961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:12.734458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:12.819045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d12c-0697-4d0a-8229-48041398a8d2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:12.823263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:12.824037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:12.824181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:12.824407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:12.834485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:12.836274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:12.847453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1895/3830] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:13.144872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:13.146574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:13.161155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cc545-0654-43e8-9f48-ebbcef01bc2f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:13.167044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:13.168490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:13.168772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:13.169043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] PATH_INFO: `/v3/users/7654d225081948cabad9d2cdbf03ebda` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:13.170273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7654d225081948cabad9d2cdbf03ebda)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:13.172246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:13.224739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7654d225081948cabad9d2cdbf03ebda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.225396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7654d225081948cabad9d2cdbf03ebda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.239425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7654d225081948cabad9d2cdbf03ebda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.243293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:13.246756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:13.258032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7654d225081948cabad9d2cdbf03ebda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.259227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7654d225081948cabad9d2cdbf03ebda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.268826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb2bb5-7965-447d-b65b-fcef1fcff8b8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:13.274386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:13.275324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:13.275705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:13.276044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] PATH_INFO: `/v3/projects/8eb4e8d763894f74868a69aac2154172` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:13.277270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb4e8d763894f74868a69aac2154172)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:13.279571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:13.289537 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1936/3831] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:13 2022] DELETE /identity/v3/users/7654d225081948cabad9d2cdbf03ebda => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:13.359727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:13.363314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb4e8d763894f74868a69aac2154172', 'request_id': 'req-e47bf9cc-363e-4806-8469-4c9ba38648e3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.371331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb4e8d763894f74868a69aac2154172', 'request_id': 'req-e47bf9cc-363e-4806-8469-4c9ba38648e3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.379659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb4e8d763894f74868a69aac2154172', 'request_id': 'req-e47bf9cc-363e-4806-8469-4c9ba38648e3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.391944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] The token cache is being invalidate because project 8eb4e8d763894f74868a69aac2154172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:13.392232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.392890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.394363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1896/3832] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:13 2022] DELETE /identity/v3/projects/8eb4e8d763894f74868a69aac2154172 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:13.398548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:13.400251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:13.420589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:13.474485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993e1e9-bd9a-4d7d-9f21-994a9531ef2a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:13.537907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:13.539411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:13.539895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:13.540358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] PATH_INFO: `/v3/users/ebc79ddabeb34c5b8bd6579250093745` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:13.542399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc79ddabeb34c5b8bd6579250093745)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:13.545444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:13.619092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc79ddabeb34c5b8bd6579250093745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.619799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc79ddabeb34c5b8bd6579250093745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.629898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc79ddabeb34c5b8bd6579250093745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.649617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc79ddabeb34c5b8bd6579250093745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.650511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1dc71266-04ce-4d1d-aea9-bf868116a7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc79ddabeb34c5b8bd6579250093745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.671311 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1937/3833] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:13 2022] DELETE /identity/v3/users/ebc79ddabeb34c5b8bd6579250093745 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:13.676357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:13.677883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:13.707429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47bf9cc-363e-4806-8469-4c9ba38648e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:13.784649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:13.785878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:13.786372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:13.786686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] PATH_INFO: `/v3/projects/e6d75184d9cd4fed89c0e1d0d5d910fd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:13.788337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d75184d9cd4fed89c0e1d0d5d910fd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:13.790429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:13.854575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d75184d9cd4fed89c0e1d0d5d910fd', 'request_id': 'req-17b934d2-0f33-4453-ad50-c9b0d60e488f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.863532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d75184d9cd4fed89c0e1d0d5d910fd', 'request_id': 'req-17b934d2-0f33-4453-ad50-c9b0d60e488f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.875543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d75184d9cd4fed89c0e1d0d5d910fd', 'request_id': 'req-17b934d2-0f33-4453-ad50-c9b0d60e488f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.894223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] The token cache is being invalidate because project e6d75184d9cd4fed89c0e1d0d5d910fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:13.894757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.895772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17b934d2-0f33-4453-ad50-c9b0d60e488f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:13.903046 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1897/3834] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:13 2022] DELETE /identity/v3/projects/e6d75184d9cd4fed89c0e1d0d5d910fd => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:13.904162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:13.904741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:13.904741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:13.940760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:13.941987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.008196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:14.056991 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1938/3835] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:14.057478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.057968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.058437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.080613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:14.081828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.102876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1898/3836] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:14.103000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.105724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d589a0-1673-458c-8d96-6eb3054d4682 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.111633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.112614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.113156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.113523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.114269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:14.114723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.116611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.131755 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1939/3837] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:14.137107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.149523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dc225-9134-4344-a7ae-352a2571fc42 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.154764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.155484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.155865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.156261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.156970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.159463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.207813 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1899/3838] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:14.208863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.221899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00494fef-a57a-4d4c-962a-f6fb3df36861 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.297505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.298295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.298678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.299014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.299736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.301833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.341854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1940/3839] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:14.355249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.372081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf7b82-71b8-4eae-bb19-1a32d2f4d841 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.387853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.388591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.389201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.391644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.394961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.397526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.421669 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1900/3840] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:14.422897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.436449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2f53-db32-4dc0-b29d-b68afddc2c31 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.443533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.444393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.444714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.445063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.445987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.447747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.478796 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1941/3841] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:14.478956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.483194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a4c7a-e401-496a-948a-26e33282b416 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.489056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.489956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.490448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.494089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] PATH_INFO: `/v3/projects/9c334364e60349c38fae5ca1915a93ef/users/aec06fda18be425bbb3fb60ddcdcb006/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.494784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.496541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.524342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245e2a-9ad2-4a3e-a2b9-7edf2d8d1c80 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.528029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c334364e60349c38fae5ca1915a93ef, user_id=aec06fda18be425bbb3fb60ddcdcb006, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.531027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb80ec-a8e7-4b5a-9888-5cd78c9d1de8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.538054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.539464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.539746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.540565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] PATH_INFO: `/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.541968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7b23feefa143999e89f42a8d3dcbc5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:14.546852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:14.555844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:14.565558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1901/3842] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:14 2022] PUT /identity/v3/projects/9c334364e60349c38fae5ca1915a93ef/users/aec06fda18be425bbb3fb60ddcdcb006/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:14.572065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.572699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.573425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.618241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:14.619476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.674406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7b23feefa143999e89f42a8d3dcbc5', 'request_id': 'req-66c42948-3ca5-4655-bfb7-cb0c63956042'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:14.693917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7b23feefa143999e89f42a8d3dcbc5', 'request_id': 'req-66c42948-3ca5-4655-bfb7-cb0c63956042'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:14.731092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7b23feefa143999e89f42a8d3dcbc5', 'request_id': 'req-66c42948-3ca5-4655-bfb7-cb0c63956042'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:14.764523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] The token cache is being invalidate because project ee7b23feefa143999e89f42a8d3dcbc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:14.764974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:14.765990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66c42948-3ca5-4655-bfb7-cb0c63956042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:14.768139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1942/3843] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:14 2022] DELETE /identity/v3/projects/ee7b23feefa143999e89f42a8d3dcbc5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:14.775507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909674c-4313-4578-9d0d-6a501a273f04 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.775884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909674c-4313-4578-9d0d-6a501a273f04 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.776229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909674c-4313-4578-9d0d-6a501a273f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.793782 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1902/3844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:14.799141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c909674c-4313-4578-9d0d-6a501a273f04 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:14.801147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c909674c-4313-4578-9d0d-6a501a273f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.806009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.807909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.869811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:14.916229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94c6a1-2e82-4bd6-944b-92ca6e81f884 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.924267 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1943/3845] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:14.932950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.933235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.934812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.960821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:14.962368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:14.975744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1944/3846] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:14.979923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:14.994951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269ac0-10f0-4d82-b3f7-ae138eee67e8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:14.995295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:14.996280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:14.996612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:14.997135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:14.999518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:15.000478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.001085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.001365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.002538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.003207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:15.003512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.005917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.017436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.019210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.029128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1945/3847] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:15.034081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.037414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1903/3848] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:04:15.049119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485848f6-ae38-4b2e-8ef1-95672bd51d37 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.054995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.055630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.056455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.056455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.057188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.059877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.092643 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1946/3849] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:15 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:15.096650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.110196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d5de3-6e8a-499e-9402-93bdd26411a1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.181008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.181766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.182120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.182489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.183222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.185356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.195764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:15.212021 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1904/3850] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:15 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:15.216611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.230471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb858ec-e77a-4b18-8f90-a23751b4900e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.237127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.238251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.238891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.239374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.240258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.243214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.262059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.270566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1947/3851] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:04:15.275968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130813b4-6d8d-4b29-bee4-8f6e08d5cc03 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.282374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.283678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.284117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.284579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.285382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.288178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da4b53-70dc-4ecc-9f60-c64108ec3738 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.307412 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1905/3852] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:15.308738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.320121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72840edd-79a8-42d0-a68e-70171812bfe3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.326436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.327124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.327304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.327579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] PATH_INFO: `/v3/projects/152924fd5da14033851c6ebc6336e8ca/users/5cc0e20c1d3142eba2bb65f18248c25b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.381881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=152924fd5da14033851c6ebc6336e8ca, user_id=5cc0e20c1d3142eba2bb65f18248c25b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.386744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.406217 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1948/3853] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:15 2022] PUT /identity/v3/projects/152924fd5da14033851c6ebc6336e8ca/users/5cc0e20c1d3142eba2bb65f18248c25b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:15.417233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.417624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.417894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.449756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] MFA Rules not processed for user `5cc0e20c1d3142eba2bb65f18248c25b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:15.451090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:15.529312 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1906/3854] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:15.534726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.545174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8445c-20f7-45c5-9a2a-386bdfd96d7c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.550010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cc0e20c1d3142eba2bb65f18248c25b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '152924fd5da14033851c6ebc6336e8ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.550800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.551111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.551363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] PATH_INFO: `/v3/domains/default` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.553094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.565044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca2b9-2e78-4329-948d-16bcc72650f2 tempest-DefaultDomainTestJSON-1658546419 tempest-DefaultDomainTestJSON-1658546419-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.567771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1949/3855] 10.222.0.52 () {58 vars in 1152 bytes} [Sat Sep 24 21:04:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:15.573458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:15.655511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e370b4-e9e7-442f-b406-6b1b1b94b8b8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:15.661350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:15.662250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.662555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.662830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] PATH_INFO: `/v3/users/5cc0e20c1d3142eba2bb65f18248c25b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.664645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc0e20c1d3142eba2bb65f18248c25b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:15.666658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:15.736523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc0e20c1d3142eba2bb65f18248c25b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:15.737866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc0e20c1d3142eba2bb65f18248c25b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:15.750360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc0e20c1d3142eba2bb65f18248c25b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:15.765053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc0e20c1d3142eba2bb65f18248c25b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:15.766014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc0e20c1d3142eba2bb65f18248c25b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:15.797798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:15.821912 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1907/3856] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:15 2022] DELETE /identity/v3/users/5cc0e20c1d3142eba2bb65f18248c25b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:15.826302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:15.826691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:15.827049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:15.863719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:15.865331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:15.971678 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1950/3857] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:16.011307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:16.012552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:16.123085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f7bd0-ae40-46a3-87ba-d6cd1c05d957 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:16.135718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:16.136347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:16.136620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:16.136886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:16.159396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:16.160858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735dfc9-c1a2-45bd-8ed8-c125d311b394 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:16.186301 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1908/3858] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:17.943490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:17.945602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:17.981087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af5ee5-2a81-400f-a9e8-99058f0b3709 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:17.988962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:17.989831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:17.990386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:17.990980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] PATH_INFO: `/v3/projects/152924fd5da14033851c6ebc6336e8ca` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:17.993718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=152924fd5da14033851c6ebc6336e8ca)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:17.996008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.082293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:18.088278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152924fd5da14033851c6ebc6336e8ca', 'request_id': 'req-c0848643-0e6b-4044-94e3-88891d0d11eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:18.096978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152924fd5da14033851c6ebc6336e8ca', 'request_id': 'req-c0848643-0e6b-4044-94e3-88891d0d11eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:18.107821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152924fd5da14033851c6ebc6336e8ca', 'request_id': 'req-c0848643-0e6b-4044-94e3-88891d0d11eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:18.126828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] The token cache is being invalidate because project 152924fd5da14033851c6ebc6336e8ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:18.127359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:18.128591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c0848643-0e6b-4044-94e3-88891d0d11eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:18.130288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1951/3859] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:17 2022] DELETE /identity/v3/projects/152924fd5da14033851c6ebc6336e8ca => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:18.135558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.136058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.136149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.161611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:18.162865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:18.238375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1909/3860] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:18.246127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.246410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.247189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.267560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:18.268357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:18.283337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.291048 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1952/3861] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:18.295474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fd029-b052-4f82-8a36-e538d008ec18 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.301078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.301794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.302167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.302550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.303379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:18.303895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.306281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.350418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.356840 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1910/3862] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:18.362567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1cd16-13ac-4e71-9b06-58ff5c03d70b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.394229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.394890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.395172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.395445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.399127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.404607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.463100 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1953/3863] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:18 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:18.463506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.479308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e151-6ddf-494b-9980-7e26df16cf42 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.603931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.604854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.605313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.605784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.606841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.609043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.653953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.656006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1911/3864] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:18 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:18.667743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb71270-bbb5-4ade-93a9-e1ed394b1dca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.673769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.674956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.675168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.675440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.676175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.677852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.699520 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1954/3865] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:18.704384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.718053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b58b0-d3f4-4b89-9837-b98dd3f127f8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.725285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.726449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.726803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.727070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.727711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.730382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6950208-b039-47e5-8e2a-72879ebfefa0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.732171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:18.746335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1912/3866] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:18.750957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:18.765957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46b5ff-ffa8-4fba-a53a-b0838a1822b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:18.791544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:18.792426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.792993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.792993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] PATH_INFO: `/v3/projects/711d576bc17e484cbf34fa7e30dabbc3/users/e13244c600b345d5b73ceb28a733a833/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:18.813881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=711d576bc17e484cbf34fa7e30dabbc3, user_id=e13244c600b345d5b73ceb28a733a833, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:18.817565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:18.860628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1955/3867] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:18 2022] PUT /identity/v3/projects/711d576bc17e484cbf34fa7e30dabbc3/users/e13244c600b345d5b73ceb28a733a833/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:18.966580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178ca4b-5b80-473f-a7a3-2ed869d90f39 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:18.967252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178ca4b-5b80-473f-a7a3-2ed869d90f39 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:18.967785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178ca4b-5b80-473f-a7a3-2ed869d90f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:19.009339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6178ca4b-5b80-473f-a7a3-2ed869d90f39 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:19.011757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6178ca4b-5b80-473f-a7a3-2ed869d90f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:19.102571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1913/3868] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:19.106930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:19.120118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd31f87-bad2-4053-a316-b18636b37e7f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:19.124378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e25d48-3350-48e2-a9ac-c4a5662fc482 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:19.125025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25d48-3350-48e2-a9ac-c4a5662fc482 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:19.125274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25d48-3350-48e2-a9ac-c4a5662fc482 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:19.125541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25d48-3350-48e2-a9ac-c4a5662fc482 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:19.184899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90e25d48-3350-48e2-a9ac-c4a5662fc482 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:19.196466 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1956/3869] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:19 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:20.213542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:20.213542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:20.213542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:20.223129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:20.224398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:20.229413 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1914/3870] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:20 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:20.234266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:20.234799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:20.234921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:20.255822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:20.258260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:20.270425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:20.274072 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1957/3871] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:20.284445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b49f19-5b19-4019-9eff-206b63e4e746 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:20.290975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:20.291859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:20.292434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:20.292949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:20.355153 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 24 21:04:20.356255 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:20.371909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:20.383695 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1915/3872] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:20 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 188 bytes in 108 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 24 21:04:20.400883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41ac78-0247-4f10-9593-b2cb1c30bdbb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:20.414305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7319c04-7430-46c0-8877-62898a7605cc tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:20.415141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7319c04-7430-46c0-8877-62898a7605cc tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:20.415424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7319c04-7430-46c0-8877-62898a7605cc tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:20.415697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7319c04-7430-46c0-8877-62898a7605cc tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b7319c04-7430-46c0-8877-62898a7605cc tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:20.491921 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. Sep 24 21:04:20.493346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:04:20.494578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1958/3873] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:20 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 188 bytes in 122 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 24 21:04:20.498938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:20.521350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08551ed8-0771-46a7-a387-6bb5dea9d8df tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:20.528025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcbe4f-cdca-4593-bbe8-d175b5bda47b tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:20.528984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcbe4f-cdca-4593-bbe8-d175b5bda47b tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:20.529433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcbe4f-cdca-4593-bbe8-d175b5bda47b tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:20.530728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcbe4f-cdca-4593-bbe8-d175b5bda47b tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:20.599603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5bcbe4f-cdca-4593-bbe8-d175b5bda47b tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:20.611190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1916/3874] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:20 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:21.620671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:21.621838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:21.622322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:21.717877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:21.724510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:21.737999 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1959/3875] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:21 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:21.743347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219edd9-8686-463b-b2c7-dd599021c9ec None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:21.743924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219edd9-8686-463b-b2c7-dd599021c9ec None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:21.743924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219edd9-8686-463b-b2c7-dd599021c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:21.807783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0219edd9-8686-463b-b2c7-dd599021c9ec None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:21.809644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0219edd9-8686-463b-b2c7-dd599021c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:21.832632 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1917/3876] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:21.837524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:21.865234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc028a-7d15-45b5-b7ed-bcb09f3da108 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:21.872684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48459-3dca-4ea4-9cfa-8402fab24bad tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:21.873693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48459-3dca-4ea4-9cfa-8402fab24bad tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:21.874239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48459-3dca-4ea4-9cfa-8402fab24bad tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:21.874718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48459-3dca-4ea4-9cfa-8402fab24bad tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:21.930037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fe48459-3dca-4ea4-9cfa-8402fab24bad tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:21.945443 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1960/3877] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:21 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:22.951626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:22.953036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:22.953492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:22.980365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:22.981889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:22.985816 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1918/3878] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:22 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:22.990372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff9be0-e446-437d-a9c9-0b813e27b1cf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:22.990939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff9be0-e446-437d-a9c9-0b813e27b1cf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:22.990939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff9be0-e446-437d-a9c9-0b813e27b1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:23.016037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-98ff9be0-e446-437d-a9c9-0b813e27b1cf None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:23.017344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98ff9be0-e446-437d-a9c9-0b813e27b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:23.026678 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1961/3879] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:23.030962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:23.047719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c62f1-67f2-4b2c-812b-f9e007bbf0e6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:23.052395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac433d3-7afb-48bc-8e71-df37821c8567 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:23.053044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac433d3-7afb-48bc-8e71-df37821c8567 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:23.053328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac433d3-7afb-48bc-8e71-df37821c8567 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:23.054368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac433d3-7afb-48bc-8e71-df37821c8567 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:23.132626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ac433d3-7afb-48bc-8e71-df37821c8567 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:23.143303 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1919/3880] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:23 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:23.858392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:23.858756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:23.859034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:23.897983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] MFA Rules not processed for user `aec06fda18be425bbb3fb60ddcdcb006`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:23.899527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:23.909171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=7,keystone:SELECT=84,keystone:INSERT=7,keystone:UPDATE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:23.990442 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1962/3881] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:23.993901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f494-e906-4a76-b21a-649b96792453 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:23.994767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f494-e906-4a76-b21a-649b96792453 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:23.995827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56f494-e906-4a76-b21a-649b96792453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:24.023926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6d56f494-e906-4a76-b21a-649b96792453 None None] MFA Rules not processed for user `aec06fda18be425bbb3fb60ddcdcb006`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:24.025220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d56f494-e906-4a76-b21a-649b96792453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:24.044962 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1920/3882] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:24.066325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:24.067925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:24.149251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccb592-f127-4e41-a1b5-1af2d0662801 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:24.149535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccb592-f127-4e41-a1b5-1af2d0662801 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:24.149947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccb592-f127-4e41-a1b5-1af2d0662801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:24.151484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966c506-5353-4de6-ba4b-7ae94c8d89b0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:24.157768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:24.158707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:24.158986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:24.159686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:24.168975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:24.170844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:24.172685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dbccb592-f127-4e41-a1b5-1af2d0662801 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:24.173864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dbccb592-f127-4e41-a1b5-1af2d0662801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:24.179531 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1921/3883] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:24 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:24.183576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0a3b-ea9f-4b11-84ff-10c979b52d82 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:24.184065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0a3b-ea9f-4b11-84ff-10c979b52d82 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:24.184548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0a3b-ea9f-4b11-84ff-10c979b52d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:24.185266 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1963/3884] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:24.210926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f8dd0a3b-ea9f-4b11-84ff-10c979b52d82 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:24.210926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f8dd0a3b-ea9f-4b11-84ff-10c979b52d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:24.226157 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1922/3885] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:24.230118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:24.240586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e26670-70bd-46c7-a92a-367ef930da4c service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:24.245859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd72158-4a63-4c2f-8c5f-7810fd7572b3 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:24.246404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd72158-4a63-4c2f-8c5f-7810fd7572b3 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:24.246726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd72158-4a63-4c2f-8c5f-7810fd7572b3 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:24.247003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd72158-4a63-4c2f-8c5f-7810fd7572b3 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:24.308563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4fd72158-4a63-4c2f-8c5f-7810fd7572b3 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:24.322670 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1964/3886] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:24 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:25.326385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326e2b-3354-45ab-942a-eb9882f0b004 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:25.326976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326e2b-3354-45ab-942a-eb9882f0b004 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:25.326976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326e2b-3354-45ab-942a-eb9882f0b004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:25.372356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0b326e2b-3354-45ab-942a-eb9882f0b004 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:25.374530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0b326e2b-3354-45ab-942a-eb9882f0b004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:25.392575 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1923/3887] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:25 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 21:04:26.398732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32bfb4-2fcc-495b-abdc-0abf7b2c76e8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:26.400201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32bfb4-2fcc-495b-abdc-0abf7b2c76e8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:26.400466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32bfb4-2fcc-495b-abdc-0abf7b2c76e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:26.430947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7c32bfb4-2fcc-495b-abdc-0abf7b2c76e8 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:26.432391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c32bfb4-2fcc-495b-abdc-0abf7b2c76e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:26.446349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1965/3888] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:26.453829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8ee5-b541-43a4-a14b-82b9ad74d08f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:26.455306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8ee5-b541-43a4-a14b-82b9ad74d08f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:26.455583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8ee5-b541-43a4-a14b-82b9ad74d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:26.488708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d34c8ee5-b541-43a4-a14b-82b9ad74d08f None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:26.492904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d34c8ee5-b541-43a4-a14b-82b9ad74d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:26.499668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1924/3889] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:26 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:26.504620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c19cd-2a26-4958-9308-63d86db3b0fc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:26.505713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c19cd-2a26-4958-9308-63d86db3b0fc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:26.506216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c19cd-2a26-4958-9308-63d86db3b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:26.565701 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.application [None req-555c19cd-2a26-4958-9308-63d86db3b0fc None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:04:26.570216 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1966/3890] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:26 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 65 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:04:26.572341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e689-8b6f-4dcd-8f50-c29a07034fde None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:26.572633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e689-8b6f-4dcd-8f50-c29a07034fde None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:26.573047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417e689-8b6f-4dcd-8f50-c29a07034fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:26.579635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:26.626438 ncdv-master-1 devstack@keystone.service[99133]: WARNING keystone.server.flask.application [None req-4417e689-8b6f-4dcd-8f50-c29a07034fde None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:04:26.628165 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1925/3891] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:26 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 57 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:04:26.635867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043611a1-7ed2-4755-a465-197932ae92eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:26.636161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043611a1-7ed2-4755-a465-197932ae92eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:26.636432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043611a1-7ed2-4755-a465-197932ae92eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:26.673112 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.application [None req-043611a1-7ed2-4755-a465-197932ae92eb None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:04:26.677082 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1967/3892] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:26 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 42 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:04:32.003513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:32.004029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:32.004029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:32.024359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:32.025574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:32.093408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1926/3893] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:36.657537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:UPDATE=2,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:37.691446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0d93-32ab-4c26-862b-cc40fb4ef998 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:37.692045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0d93-32ab-4c26-862b-cc40fb4ef998 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:37.692045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe0d93-32ab-4c26-862b-cc40fb4ef998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:37.736413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3ffe0d93-32ab-4c26-862b-cc40fb4ef998 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:37.737647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ffe0d93-32ab-4c26-862b-cc40fb4ef998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:37.740745 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1968/3894] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:37 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:37.745991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:37.756002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d526ce-b06e-4383-99ca-379b27146a26 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:37.760670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9bc7d2-d1ee-471f-b496-456356d4c1b1 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:37.761287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bc7d2-d1ee-471f-b496-456356d4c1b1 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:37.761567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bc7d2-d1ee-471f-b496-456356d4c1b1 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:37.761837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9bc7d2-d1ee-471f-b496-456356d4c1b1 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:37.806936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e9bc7d2-d1ee-471f-b496-456356d4c1b1 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:37.816252 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1927/3895] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:37 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:38.821785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b895-0e10-486b-b3b7-a2363e1c5991 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:38.822299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b895-0e10-486b-b3b7-a2363e1c5991 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:38.822407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3b895-0e10-486b-b3b7-a2363e1c5991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:38.852131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-56b3b895-0e10-486b-b3b7-a2363e1c5991 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:38.852131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56b3b895-0e10-486b-b3b7-a2363e1c5991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:38.855389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1969/3896] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:38 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:38.860408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c7c0-7e8d-4fe1-9385-8bb78ecd07aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:38.861438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c7c0-7e8d-4fe1-9385-8bb78ecd07aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:38.861747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c7c0-7e8d-4fe1-9385-8bb78ecd07aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-fe08c7c0-7e8d-4fe1-9385-8bb78ecd07aa None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(*args, **kwargs) Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post Sep 24 21:04:38.900678 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:38.902818 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:04:38.904167 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1928/3897] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:38 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:04:38.906521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:38.907167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:38.907167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:38.948495 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.application [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:04:38.949920 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1970/3898] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:38 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 44 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:04:38.954623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847a483-0abd-4b4a-9345-5db76d01bb34 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:38.954986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847a483-0abd-4b4a-9345-5db76d01bb34 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:38.955430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847a483-0abd-4b4a-9345-5db76d01bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:38.990866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5847a483-0abd-4b4a-9345-5db76d01bb34 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:38.992950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5847a483-0abd-4b4a-9345-5db76d01bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:39.000659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1929/3899] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:39.004620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:39.016889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506687e7-51d7-497f-98cb-196b280661a0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:39.021502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345072e-8057-4aed-a1b7-37e9b116ff24 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:39.022380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345072e-8057-4aed-a1b7-37e9b116ff24 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:39.022763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345072e-8057-4aed-a1b7-37e9b116ff24 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:39.023136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345072e-8057-4aed-a1b7-37e9b116ff24 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:39.071433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a345072e-8057-4aed-a1b7-37e9b116ff24 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:39.082705 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1971/3900] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:39 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:40.088179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90137bbe-14db-4824-b788-ee067045c961 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:40.088723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90137bbe-14db-4824-b788-ee067045c961 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:40.088866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90137bbe-14db-4824-b788-ee067045c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:40.111347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-90137bbe-14db-4824-b788-ee067045c961 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:40.113011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90137bbe-14db-4824-b788-ee067045c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:40.136162 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1930/3901] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:40 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:40.142039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a545-d691-4d4e-a27c-0fdc441f53b6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:40.142651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a545-d691-4d4e-a27c-0fdc441f53b6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:40.143084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a545-d691-4d4e-a27c-0fdc441f53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:40.169271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-86a5a545-d691-4d4e-a27c-0fdc441f53b6 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:40.171041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86a5a545-d691-4d4e-a27c-0fdc441f53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:40.179157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1972/3902] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:40.184419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90137bbe-14db-4824-b788-ee067045c961 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:40.196438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90137bbe-14db-4824-b788-ee067045c961 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:40.200551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752022ea-b782-4166-a29b-8692fccd90aa tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:40.201226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752022ea-b782-4166-a29b-8692fccd90aa tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:40.201508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752022ea-b782-4166-a29b-8692fccd90aa tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:40.201780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752022ea-b782-4166-a29b-8692fccd90aa tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:40.257083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-752022ea-b782-4166-a29b-8692fccd90aa tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:40.267842 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1931/3903] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:40 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:04:41.274221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:41.275205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:41.275576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:41.297587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:41.298869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:41.303288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1973/3904] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:41 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:41.307934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1fcb-459d-4719-b29d-674d29591823 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:41.308410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1fcb-459d-4719-b29d-674d29591823 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:41.308517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1fcb-459d-4719-b29d-674d29591823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:41.332022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-826f1fcb-459d-4719-b29d-674d29591823 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:41.333467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-826f1fcb-459d-4719-b29d-674d29591823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:41.343921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1932/3905] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:41.348742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:41.361064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab457df-0b36-403c-97a6-5c258db047aa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:41.366194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93496f81-edeb-4206-af64-ff54f68a02e9 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e13244c600b345d5b73ceb28a733a833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '711d576bc17e484cbf34fa7e30dabbc3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:41.367170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496f81-edeb-4206-af64-ff54f68a02e9 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:41.367676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496f81-edeb-4206-af64-ff54f68a02e9 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:41.368255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93496f81-edeb-4206-af64-ff54f68a02e9 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833/password` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:41.477394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-93496f81-edeb-4206-af64-ff54f68a02e9 tempest-IdentityV3UsersTest-95839356 tempest-IdentityV3UsersTest-95839356-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:41.486676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1974/3906] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:04:41 2022] POST /identity/v3/users/e13244c600b345d5b73ceb28a733a833/password => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:42.492780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:42.493243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:42.493452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:42.518620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:42.519826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:42.525679 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1933/3907] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:42 2022] POST /identity/v3/auth/tokens => generated 351 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:04:43.529906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45363b-aeb2-4a62-beb3-f7b775efc325 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:43.530456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45363b-aeb2-4a62-beb3-f7b775efc325 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:43.530719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45363b-aeb2-4a62-beb3-f7b775efc325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:43.552081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5e45363b-aeb2-4a62-beb3-f7b775efc325 None None] MFA Rules not processed for user `e13244c600b345d5b73ceb28a733a833`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:43.553308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5e45363b-aeb2-4a62-beb3-f7b775efc325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:43.565555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1975/3908] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:43.568462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:43.621103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:UPDATE=4,keystone:INSERT=4,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:43.661202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b8262-a04f-4d0c-b6c1-a85ab292c008 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:43.667943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:43.668586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:43.668866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:43.669144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] PATH_INFO: `/v3/users/e13244c600b345d5b73ceb28a733a833` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:43.670484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13244c600b345d5b73ceb28a733a833)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:43.672719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:43.722618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:43.723276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:43.731183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:43.738966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:43.739655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13244c600b345d5b73ceb28a733a833'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:43.757966 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1934/3909] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:43 2022] DELETE /identity/v3/users/e13244c600b345d5b73ceb28a733a833 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:43.762620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:43.762906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:43.763163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:43.783615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:43.784825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:43.870685 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1976/3910] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:43.871962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:43.874219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:43.966822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f3179-fd8f-47e4-ac89-9aed205bc0d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:43.971901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:43.973939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:43.974535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:43.975004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:43.990105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:43.993704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0deb561-1e91-4a17-ba92-98efe9741b3a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:44.011260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1935/3911] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:44.417522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:44.419162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:44.431710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bebf5-4343-406c-bf8b-cf63b7e4eca1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:44.437813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:44.439067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:44.439381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:44.439765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] PATH_INFO: `/v3/projects/711d576bc17e484cbf34fa7e30dabbc3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:44.442669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711d576bc17e484cbf34fa7e30dabbc3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:44.444258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:44.469395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:UPDATE=4,keystone:INSERT=4,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:44.524025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711d576bc17e484cbf34fa7e30dabbc3', 'request_id': 'req-90cc79e1-7055-4209-a3c8-52c3b8caf827'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:44.538742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711d576bc17e484cbf34fa7e30dabbc3', 'request_id': 'req-90cc79e1-7055-4209-a3c8-52c3b8caf827'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:44.547857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711d576bc17e484cbf34fa7e30dabbc3', 'request_id': 'req-90cc79e1-7055-4209-a3c8-52c3b8caf827'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:44.567203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] The token cache is being invalidate because project 711d576bc17e484cbf34fa7e30dabbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:44.567498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:44.568750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-90cc79e1-7055-4209-a3c8-52c3b8caf827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:44.572122 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1977/3912] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:44 2022] DELETE /identity/v3/projects/711d576bc17e484cbf34fa7e30dabbc3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:44.578102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:44.578604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:44.578937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:44.604600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:44.606141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:44.749666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:44.773916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:44.774658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:44.775116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:44.783244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1936/3913] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:44.803901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:44.806251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:44.816810 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1978/3914] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:44.820903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:44.840284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60ad3e-7c57-4bba-addd-6dc0cc1ef242 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:44.846239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:44.847100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:44.848425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:44.848860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:44.850005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:44.850814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:44.854641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:44.875109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:44.881503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1937/3915] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:44.886728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa482-f3ba-416f-93a0-44756443cba0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:44.895418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:44.896404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:44.896976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:44.897724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:44.898755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:44.901305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:44.963842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1979/3916] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:44 2022] POST /identity/v3/projects => generated 391 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:44.965274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:44.986623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f4994-e690-4f52-850f-451ab8983ef5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.113111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.113884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.114181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.114547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.115258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.118335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.146627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1938/3917] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:44 2022] POST /identity/v3/users => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:45.149630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.161315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fc2ea-f50c-42b7-9cdf-1c81d2bb97e2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.167041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.167770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.167943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.168418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.169062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.170919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.181640 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1980/3918] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:04:45.186091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.197320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cab5d-cf2c-4c25-b439-4aa86dc10683 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.203855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.204523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.204860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.205129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] PATH_INFO: `/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893/users/8d9638b2a0be438e89d5a55799e40e47/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.228097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbb5e75b1de4d37a5042de7a1da8893, user_id=8d9638b2a0be438e89d5a55799e40e47, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.231089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.248832 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1939/3919] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:45 2022] PUT /identity/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893/users/8d9638b2a0be438e89d5a55799e40e47/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:45.252911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.263932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7fd43-46bd-4d8f-8e36-21b3a521999d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.360389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.361181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.361657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.362159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.363228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.364890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82181188-2904-4968-9a3c-95e017eafd08 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.391716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.393698 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1981/3920] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:45.404635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf68f7-0b2d-46ee-b15f-f812ce3265b3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.419026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.419026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.419356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.419623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] PATH_INFO: `/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893/users/8d9638b2a0be438e89d5a55799e40e47/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.422096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbb5e75b1de4d37a5042de7a1da8893, user_id=8d9638b2a0be438e89d5a55799e40e47, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.425054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.450986 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1940/3921] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:45 2022] PUT /identity/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893/users/8d9638b2a0be438e89d5a55799e40e47/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:45.458384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.458701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.459036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.506534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] MFA Rules not processed for user `8d9638b2a0be438e89d5a55799e40e47`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:45.507828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:45.567886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:45.597918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1982/3922] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:45.602989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.676157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c462-933a-48e1-bab2-b9f69d11207f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.681855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.682568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.682953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.683232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.683878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.686171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.733996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1941/3923] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:45 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:45.735334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.746920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb92dfc-0e2a-4816-b5fc-cfe9e635c5e0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.824619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.825657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.826344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.827116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.828080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.830373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.857264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1983/3924] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:45 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:45.858617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.869944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f4617-7fa1-458a-aefe-e128c9f4ed76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.875769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.876519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.876928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.877208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.877849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.879500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.894231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1942/3925] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:45.895711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.912909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f2984-48c5-420f-a2fa-c4ea05048f1b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.920146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.920768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.921043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.921318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.922184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.923839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569e617-2777-4887-91c9-15dadab60b8d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.936008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1984/3926] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:45.940373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:45.954020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd28ddc-9fd2-4091-a297-646a23f00159 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:45.958859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:45.959525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:45.959802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:45.960075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] PATH_INFO: `/v3/projects/1c173bf1c0d84cfcacc91a1efb970881/users/230c594bfbc1435c8141582c50435197/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:45.975753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:45.977209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c173bf1c0d84cfcacc91a1efb970881, user_id=230c594bfbc1435c8141582c50435197, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:45.980116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:45.996707 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1943/3927] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:45 2022] PUT /identity/v3/projects/1c173bf1c0d84cfcacc91a1efb970881/users/230c594bfbc1435c8141582c50435197/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:46.008492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc31b5-408c-400c-9b9e-4e7367006f49 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:46.008787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc31b5-408c-400c-9b9e-4e7367006f49 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:46.009059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc31b5-408c-400c-9b9e-4e7367006f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:46.041225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f0dc31b5-408c-400c-9b9e-4e7367006f49 None None] MFA Rules not processed for user `230c594bfbc1435c8141582c50435197`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:46.042627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f0dc31b5-408c-400c-9b9e-4e7367006f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:46.174233 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1985/3928] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:46.195083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:46.195083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:46.350820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9a648-f344-4156-843e-080f27037f8d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:46.363512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:46.364276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:46.364622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:46.364961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:46.480862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:46.482235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:46.506846 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1944/3929] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:46.915603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:46.916072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:46.916279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:46.967414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:46.973285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:47.073230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:47.090675 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1986/3930] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:47.779931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:47.781409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:47.862914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e090e-3b96-4fe0-ac84-407089a46ed7 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:47.880010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:47.882086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:47.884311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:47.885437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:47.889425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:47.964745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:48.032296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:48.034109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:48.047920 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1945/3931] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:48.062590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:48.170248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:48.171801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:48.213612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c2138-719e-4b0a-bd4c-2747a538e417 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:48.218841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:48.219477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.219962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.220253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] PATH_INFO: `/v3/users/8d9638b2a0be438e89d5a55799e40e47` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.221386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9638b2a0be438e89d5a55799e40e47)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:48.224314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:48.267798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc30101-ec18-4121-8b3a-b1c9a7d8063b service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:48.274816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:48.275568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.275853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.276188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] PATH_INFO: `/v3/users/aec06fda18be425bbb3fb60ddcdcb006` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.277418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec06fda18be425bbb3fb60ddcdcb006)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:48.279764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:48.292727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9638b2a0be438e89d5a55799e40e47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.293400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9638b2a0be438e89d5a55799e40e47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.302311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9638b2a0be438e89d5a55799e40e47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.330261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9638b2a0be438e89d5a55799e40e47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.331225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a5a8bfb-4d45-40d6-b609-a041d69ccd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9638b2a0be438e89d5a55799e40e47'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.365911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1987/3932] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:48 2022] DELETE /identity/v3/users/8d9638b2a0be438e89d5a55799e40e47 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:48.372744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec06fda18be425bbb3fb60ddcdcb006'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.373156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.373447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.373721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.374907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec06fda18be425bbb3fb60ddcdcb006'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.388438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec06fda18be425bbb3fb60ddcdcb006'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.406698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:48.406698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:48.412661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec06fda18be425bbb3fb60ddcdcb006'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.414050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec06fda18be425bbb3fb60ddcdcb006'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.454850 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1946/3933] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:48 2022] DELETE /identity/v3/users/aec06fda18be425bbb3fb60ddcdcb006 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:48.515757 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1988/3934] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:48.537309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:48.538949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:48.628193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:48.672360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd5aa-6a9e-4ffa-bb99-71f522b9707f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:48.677585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:48.678662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.680514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.680909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.728137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:48.730129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:48.768062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ae37e-b7f0-45ed-9d30-06d62050d000 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:48.794993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:48.801802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:48.803381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c5f1f-a40e-4c5b-9902-f036edae327d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:48.816597 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1947/3935] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:48.880423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:48.881060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.881466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.881742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] PATH_INFO: `/v3/projects/9c334364e60349c38fae5ca1915a93ef` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.883639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c334364e60349c38fae5ca1915a93ef)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:48.886798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:48.938882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c334364e60349c38fae5ca1915a93ef', 'request_id': 'req-57d8af01-e59c-4a59-8034-b2322f677a48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.945492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c334364e60349c38fae5ca1915a93ef', 'request_id': 'req-57d8af01-e59c-4a59-8034-b2322f677a48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.952703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c334364e60349c38fae5ca1915a93ef', 'request_id': 'req-57d8af01-e59c-4a59-8034-b2322f677a48'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.965841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] The token cache is being invalidate because project 9c334364e60349c38fae5ca1915a93ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:48.966125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.967376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57d8af01-e59c-4a59-8034-b2322f677a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:48.969298 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1989/3936] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:48 2022] DELETE /identity/v3/projects/9c334364e60349c38fae5ca1915a93ef => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:48.974003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:48.974290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:48.974607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:48.996730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:48.997955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.061536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1948/3937] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:49.069507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.070749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.070878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.091650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:49.092937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.101836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1990/3938] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:49.106555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.117490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace9a44-1653-4bf8-8ceb-0cbae45a2c10 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.122152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.122793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.123075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.123347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.124003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:49.124389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.125898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.142464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1949/3939] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:04:49.143787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.153810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dced3-0e67-4667-9fbd-47d67054e4e9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.159286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.159898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.160464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.160956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.161897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.164932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.216345 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1991/3940] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:49 2022] POST /identity/v3/projects => generated 401 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:49.217624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.230925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37924fb-1b83-4af0-80ce-c5e4efda833f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.315130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.316081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.316081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.316318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.316853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.319081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.361573 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1950/3941] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:49 2022] POST /identity/v3/users => generated 301 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:49.363739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.378604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1fc64-4327-4c3e-8220-59f098c36e75 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.391539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.392476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.393400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.393986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.395964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.397839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.421113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.422644 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1992/3942] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:49.436036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef012bfe-696e-47d0-a69f-26159dd91b70 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.450408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.451776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.454287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.454893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.455991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.459433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.472996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.478107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:49.482538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.507254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd655-2ba2-49c8-89b6-31109c8d7c98 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.514719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.516307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.517386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.517809 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1951/3943] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:49.518292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] PATH_INFO: `/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.519406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.519934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbb5e75b1de4d37a5042de7a1da8893)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.522122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.532658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6f20-adb4-4462-9398-8bea894c46d9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.537923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.540096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.540419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.540698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] PATH_INFO: `/v3/projects/3ee2eb7dffa647fb89edd0cf7d1cb454/users/f05599f233ca45c39ccd351c0432f1ba/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.575798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee2eb7dffa647fb89edd0cf7d1cb454, user_id=f05599f233ca45c39ccd351c0432f1ba, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.578438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf276f-d5e5-4695-b8cf-a2db9d1a7317 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.599517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1952/3944] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:49 2022] PUT /identity/v3/projects/3ee2eb7dffa647fb89edd0cf7d1cb454/users/f05599f233ca45c39ccd351c0432f1ba/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:49.601811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbb5e75b1de4d37a5042de7a1da8893', 'request_id': 'req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.605400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.606893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.607988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.608754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbb5e75b1de4d37a5042de7a1da8893', 'request_id': 'req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.616277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbb5e75b1de4d37a5042de7a1da8893', 'request_id': 'req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.620512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:49.634385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:49.635807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.644455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] The token cache is being invalidate because project ffbb5e75b1de4d37a5042de7a1da8893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:49.645021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.645771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.648690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1993/3945] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:49 2022] DELETE /identity/v3/projects/ffbb5e75b1de4d37a5042de7a1da8893 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:49.653412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.656356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.705222 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1953/3946] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:49.727005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:49.727005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:49.770933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0d32d-a7c6-4e4a-998e-64b7f39aee84 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.785293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.785947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.787599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.787599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] PATH_INFO: `/v3/users/230c594bfbc1435c8141582c50435197` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.789194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230c594bfbc1435c8141582c50435197)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.791374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.808642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9353f-f942-4ea9-8057-5f15d2d4027e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:49.814800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:49.815873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:49.816635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:49.817101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:49.830717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:49.833523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:49.853319 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1954/3947] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:49.877957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230c594bfbc1435c8141582c50435197'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.879543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230c594bfbc1435c8141582c50435197'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.890040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230c594bfbc1435c8141582c50435197'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.911224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230c594bfbc1435c8141582c50435197'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.912203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e811f962-36ad-4720-9355-b8cc81ce6850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230c594bfbc1435c8141582c50435197'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:49.932834 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1994/3948] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:49 2022] DELETE /identity/v3/users/230c594bfbc1435c8141582c50435197 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:50.385230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:50.415133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4843-6eef-41dd-abe5-7d2141d0d54f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:50.534079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:50.534812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:50.535102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:50.535380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] PATH_INFO: `/v3/projects/1c173bf1c0d84cfcacc91a1efb970881` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:50.536788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c173bf1c0d84cfcacc91a1efb970881)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:50.541389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:50.566435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:50.634352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c173bf1c0d84cfcacc91a1efb970881', 'request_id': 'req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:50.660944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c173bf1c0d84cfcacc91a1efb970881', 'request_id': 'req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:50.684067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c173bf1c0d84cfcacc91a1efb970881', 'request_id': 'req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:50.721219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] The token cache is being invalidate because project 1c173bf1c0d84cfcacc91a1efb970881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:50.722354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:50.723355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-770fbb4d-c407-4482-ad3f-7ca0a225b7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:50.726661 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1955/3949] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:50 2022] DELETE /identity/v3/projects/1c173bf1c0d84cfcacc91a1efb970881 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:50.731461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:50.731939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:50.732044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:50.760067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:50.763606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:50.900604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1995/3950] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:50.904156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:50.904687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:50.905531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:50.936133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:50.937396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:50.948634 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1956/3951] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:50.954705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:50.969633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d0058-f71e-40bf-8283-4ca83483db1e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:50.984482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:50.985111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:50.985389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:50.985660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:50.986316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:50.986645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:50.988244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.011625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:51.020287 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1996/3952] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:51.022841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.040589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c296af-f0fc-46bb-9e45-a7d6915492f3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.045436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.046040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.046316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.046701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.047947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.049826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.084686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1957/3953] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:51 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:51.089145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.104859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fad8-7821-4cd9-9c72-eb27d181fe88 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.179741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.180368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.180671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.183470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.183806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.185918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.220316 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1997/3954] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:51 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:51.221321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.237748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9b581-00a8-4786-94ee-6816b67fa73b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.244655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.245715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.245715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.246691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.247955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.249915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.267304 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1958/3955] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:04:51.273844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.300256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fd320-d753-4359-a3d7-e62f3df14493 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.308004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.308833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.309259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.309633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] PATH_INFO: `/v3/projects/3922489e66aa41a2841786858097c6eb/users/df0fa010be284e1891f6caa128155f77/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.339125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3922489e66aa41a2841786858097c6eb, user_id=df0fa010be284e1891f6caa128155f77, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.342153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.371381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1998/3956] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:51 2022] PUT /identity/v3/projects/3922489e66aa41a2841786858097c6eb/users/df0fa010be284e1891f6caa128155f77/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:51.376668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.393808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd48c3-9735-441e-a10c-d6954f5f6f3e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.517022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.518093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.518692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.519207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.522549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.525796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8944b6-66ad-4053-998c-41dc6398fdac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.547798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:51.553212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1959/3957] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:51.559777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0346a-c7b9-46b0-a751-64b9b7dcdaf5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:51.568835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:51.569180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.570444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.570444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] PATH_INFO: `/v3/projects/3922489e66aa41a2841786858097c6eb/users/df0fa010be284e1891f6caa128155f77/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.576610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3922489e66aa41a2841786858097c6eb, user_id=df0fa010be284e1891f6caa128155f77, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:51.580045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:51.609382 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 1999/3958] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:51 2022] PUT /identity/v3/projects/3922489e66aa41a2841786858097c6eb/users/df0fa010be284e1891f6caa128155f77/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:51.627858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:51.628151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:51.628452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:51.718894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] MFA Rules not processed for user `df0fa010be284e1891f6caa128155f77`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:51.720149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:51.867999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:51.925663 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1960/3959] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:51.931120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.064372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b002d5-8114-4e63-8590-c1f6486c9b12 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.070447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.071086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.071364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.071633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.072268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.074189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.096939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.098761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:52.142092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2000/3960] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:51 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:52.143657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.249355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafb51-c36e-4c72-bfaa-30f9335366da None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.255108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:52.309767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792adae-1e9f-439b-9c46-024feb4a759f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.323564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.324333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.324773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.325275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.326047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.328368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.372358 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2001/3961] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:52 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:04:52.376948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.393731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.394801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.395088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.395362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.396864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e8ddd-cfc7-4187-9cc0-2f66ec20e9cd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.398249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:52.411134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f3273-23c5-411b-9529-76841536020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.411939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f3273-23c5-411b-9529-76841536020e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.412325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f3273-23c5-411b-9529-76841536020e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.412703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f3273-23c5-411b-9529-76841536020e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.414617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f3273-23c5-411b-9529-76841536020e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.416389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f3273-23c5-411b-9529-76841536020e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.446288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2002/3962] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:52.450885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f3273-23c5-411b-9529-76841536020e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.471388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f3273-23c5-411b-9529-76841536020e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.477876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.478828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.479584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.480066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.481995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.484220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.498176 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2003/3963] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:52.502059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.515938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4597093-ed27-4303-89fc-df86d19f0204 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:52.521641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:52.522882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.523276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.523829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] PATH_INFO: `/v3/projects/0e58afcfb99a447e97900cc67214a728/users/1b296d59e5a94702b474b66b8499eec7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.543107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e58afcfb99a447e97900cc67214a728, user_id=1b296d59e5a94702b474b66b8499eec7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.546947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec363c-2783-4398-b446-3b2dc8368e23 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.576280 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2004/3964] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:52 2022] PUT /identity/v3/projects/0e58afcfb99a447e97900cc67214a728/users/1b296d59e5a94702b474b66b8499eec7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:52.588561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:52.589005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:52.589395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:52.623803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:52.649559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:52.655779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] MFA Rules not processed for user `1b296d59e5a94702b474b66b8499eec7`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:52.658370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:52.692828 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1961/3965] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:52.889191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2005/3966] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:52.902808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:52.904933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:52.971766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:53.000125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11c0a8-670d-4655-abe7-5d2d2ce82cb8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:53.080700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:53.081601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:53.082208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:53.082727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:53.209716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:53.212135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a267f89-18b0-48dc-824f-8ed69b2358d7 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:53.244512 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1962/3967] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:53.601527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:53.746952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c4133-26fb-46a0-a2d5-bfc5f471be6a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:53.752682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:53.753415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:53.753884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:53.754159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] PATH_INFO: `/v3/users/df0fa010be284e1891f6caa128155f77` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:53.755825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0fa010be284e1891f6caa128155f77)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:53.759913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:53.778389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:53.862987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0fa010be284e1891f6caa128155f77'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:53.864524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0fa010be284e1891f6caa128155f77'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:53.884699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0fa010be284e1891f6caa128155f77'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:53.920222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0fa010be284e1891f6caa128155f77'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:53.921052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0fa010be284e1891f6caa128155f77'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:53.954705 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2006/3968] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:53 2022] DELETE /identity/v3/users/df0fa010be284e1891f6caa128155f77 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:53.957221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:53.957505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:53.957869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:54.008375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:54.009634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:54.144179 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1963/3969] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:54.183898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:54.186093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:54.298960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ae5-f5c4-45e3-a77f-946bc94bff7f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:54.303056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:54.304445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:54.304445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:54.304445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:54.324202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:54.326282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:54.341562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2007/3970] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:54.939030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:54.941944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:54.958247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfbcd9-8119-4313-a053-9dedab00ad32 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:54.964548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:54.965815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:54.966359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:54.966684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] PATH_INFO: `/v3/projects/3922489e66aa41a2841786858097c6eb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:54.968422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3922489e66aa41a2841786858097c6eb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:54.969829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:55.045180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:55.055766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3922489e66aa41a2841786858097c6eb', 'request_id': 'req-9543a858-78cf-4cd7-8597-1ee58de90ee8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.081229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3922489e66aa41a2841786858097c6eb', 'request_id': 'req-9543a858-78cf-4cd7-8597-1ee58de90ee8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.094029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3922489e66aa41a2841786858097c6eb', 'request_id': 'req-9543a858-78cf-4cd7-8597-1ee58de90ee8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.124361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] The token cache is being invalidate because project 3922489e66aa41a2841786858097c6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:55.129076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.130072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.132732 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1964/3971] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:54 2022] DELETE /identity/v3/projects/3922489e66aa41a2841786858097c6eb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:55.136980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:55.150270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2dc45-7f75-4d70-851d-8d3a76d1689f service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:55.242428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:55.243143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:55.243423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:55.243698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] PATH_INFO: `/v3/users/1b296d59e5a94702b474b66b8499eec7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:55.244856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b296d59e5a94702b474b66b8499eec7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:55.249347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:55.329222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:55.352394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b296d59e5a94702b474b66b8499eec7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.353064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b296d59e5a94702b474b66b8499eec7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.370530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b296d59e5a94702b474b66b8499eec7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.413793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b296d59e5a94702b474b66b8499eec7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.415655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ab7a0e2-60ed-4029-9b36-8e70ac429f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b296d59e5a94702b474b66b8499eec7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:55.436467 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2008/3972] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:55 2022] DELETE /identity/v3/users/1b296d59e5a94702b474b66b8499eec7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:55.807272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:55.808798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:55.822914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543a858-78cf-4cd7-8597-1ee58de90ee8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:55.902907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:55.903565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:55.903848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:55.904124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] PATH_INFO: `/v3/projects/0e58afcfb99a447e97900cc67214a728` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:55.905384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e58afcfb99a447e97900cc67214a728)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:55.907771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.005344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e58afcfb99a447e97900cc67214a728', 'request_id': 'req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:56.020848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e58afcfb99a447e97900cc67214a728', 'request_id': 'req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:56.036761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e58afcfb99a447e97900cc67214a728', 'request_id': 'req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:56.058004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] The token cache is being invalidate because project 0e58afcfb99a447e97900cc67214a728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:56.058613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:56.059993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e1642098-361b-4dc8-a9eb-5abb4f98dc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:56.062155 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1965/3973] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:55 2022] DELETE /identity/v3/projects/0e58afcfb99a447e97900cc67214a728 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:56.070600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.070964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.071250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.122482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:56.126414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:56.237225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2009/3974] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:56.239201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.239679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.240096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.265004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:56.266384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:56.277739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1966/3975] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:56.282227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.296360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84635e02-4dd7-4e3a-89f2-9107f5682148 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.310797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.311435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.311890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.312230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.313133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:56.313445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.315343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.334148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2010/3976] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:56.343031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.370540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97a71-059c-43ec-9175-e0ac1de3c346 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.387856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.391270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.391546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.391822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.392476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.394438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.447756 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1967/3977] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:56 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:56.452442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.464826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dc056-b2d9-4f08-9e9a-4ee814c12572 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.558640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.559384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.559662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.559924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.560557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.562529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.590341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2011/3978] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:56 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:56.591582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.603183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16516894-e076-44a5-812c-31d5ba50781c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.608676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.609502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.609839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.610207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.610897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.612612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.625827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1968/3979] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:56.631942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.651072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01b034-8aec-43a4-9b33-c61a3cff9233 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.658192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.658839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.659120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.659470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.660463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.661809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36ef4a-f084-4185-a8ff-bd3e7f03506e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.674982 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2012/3980] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:04:56.680125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:56.691149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee921981-024b-4137-b40f-96860f525b0d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:56.698017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:56.699904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.699904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.699904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] PATH_INFO: `/v3/projects/01e6f09c239949069f593455e6a18c89/users/37236391bbce46d4bd9d35466c2b83c3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.735546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e6f09c239949069f593455e6a18c89, user_id=37236391bbce46d4bd9d35466c2b83c3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:56.738920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:56.750961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:56.770990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1969/3981] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:04:56 2022] PUT /identity/v3/projects/01e6f09c239949069f593455e6a18c89/users/37236391bbce46d4bd9d35466c2b83c3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:56.774032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c69810-a467-4d63-9c66-a363439029af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:56.774280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c69810-a467-4d63-9c66-a363439029af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:56.775256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c69810-a467-4d63-9c66-a363439029af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:56.813124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f2c69810-a467-4d63-9c66-a363439029af None None] MFA Rules not processed for user `37236391bbce46d4bd9d35466c2b83c3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:56.831602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2c69810-a467-4d63-9c66-a363439029af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:56.838425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:56.993358 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2013/3982] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:57.004446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:57.006362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:57.083124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea836b-fe07-427a-98a0-4abb6f9890ba admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:57.088558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:57.089201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:57.089480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:57.089754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:57.099409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:57.101463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac0722-5847-4a54-841f-633f36ebfa26 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:57.111704 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1970/3983] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:57.440856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c69810-a467-4d63-9c66-a363439029af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:57.510727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c69810-a467-4d63-9c66-a363439029af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:57.515481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:57.516142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:57.516421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:57.516689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] PATH_INFO: `/v3/users/37236391bbce46d4bd9d35466c2b83c3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:57.517809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=37236391bbce46d4bd9d35466c2b83c3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:57.519783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:57.571948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37236391bbce46d4bd9d35466c2b83c3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:57.572627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37236391bbce46d4bd9d35466c2b83c3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:57.580899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37236391bbce46d4bd9d35466c2b83c3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:57.594294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37236391bbce46d4bd9d35466c2b83c3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:57.595133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37236391bbce46d4bd9d35466c2b83c3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:57.608434 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2014/3984] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:04:57 2022] DELETE /identity/v3/users/37236391bbce46d4bd9d35466c2b83c3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:57.612860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:57.613144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:57.613416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:57.637796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:57.639014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:57.714741 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1971/3985] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:57.724570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:57.726119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:57.764373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:57.796071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d123-0e81-490a-a18c-31851d68bd7f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:57.800143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:57.800869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:57.802873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:57.803225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:57.813971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:57.815523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaee48-f174-498e-a88b-cc3dadd53558 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:57.827212 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2015/3986] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:04:59.274063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:59.275641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:59.289156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abce5e-fa14-4050-a4f9-e474cea8d6f7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:59.301092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:59.301092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:59.301092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:59.301092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] PATH_INFO: `/v3/projects/01e6f09c239949069f593455e6a18c89` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:59.304828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e6f09c239949069f593455e6a18c89)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:59.305172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:59.387153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e6f09c239949069f593455e6a18c89', 'request_id': 'req-524511a2-f0dd-41b0-b8de-bb95cf243f4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:59.403196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e6f09c239949069f593455e6a18c89', 'request_id': 'req-524511a2-f0dd-41b0-b8de-bb95cf243f4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:59.413756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e6f09c239949069f593455e6a18c89', 'request_id': 'req-524511a2-f0dd-41b0-b8de-bb95cf243f4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:59.444301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] The token cache is being invalidate because project 01e6f09c239949069f593455e6a18c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:04:59.445514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:59.445807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-524511a2-f0dd-41b0-b8de-bb95cf243f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:04:59.453094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1972/3987] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:04:59 2022] DELETE /identity/v3/projects/01e6f09c239949069f593455e6a18c89 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:04:59.453292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:59.453737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:59.453737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:59.499163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:59.500391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:59.696717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2016/3988] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:04:59.704637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:59.704920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:59.705217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:59.745751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:04:59.747026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:04:59.765900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1973/3989] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:04:59.770928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:59.784769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b0ce3-47b3-4632-8f8b-256f24c8c467 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:59.791339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:59.792082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:59.792822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:59.792822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:59.793457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:04:59.793927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:59.795644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:59.814710 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2017/3990] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:04:59.818915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:59.834763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fcd6e-2a1d-4592-b8ea-f980268bfcbb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:04:59.841622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:04:59.843127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:04:59.843280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:04:59.843584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:04:59.844690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:04:59.846822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:04:59.881167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:04:59.933527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1974/3991] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:04:59 2022] POST /identity/v3/projects => generated 409 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:04:59.938250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:04:59.963955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc2bdb-5db7-440f-b2bc-635e8b7dc023 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.096428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.097433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.097433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.097709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.098239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.102344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.140876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.148113 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2018/3992] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:04:59 2022] POST /identity/v3/users => generated 305 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:00.154047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a87f9-28b7-492d-8879-b8d1fb7a001e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.160499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.161116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.161397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.161927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.162630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.164504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.177560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1975/3993] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:00.182706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.198248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b450b-0b4a-4081-9e98-d4647f7af500 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.208128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.208947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.209298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.209661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.210955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.221256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2500cb-4ecd-4459-8995-ce2e5797b472 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.242078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2019/3994] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:00.247324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.263973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4021-ba0d-4968-a5db-57bb3724d19b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.275203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.276049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.276392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.276782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] PATH_INFO: `/v3/projects/aa118e5626f644b18b955a64e1ef34db/users/5bf30f9e6682405692158a3b7a29d0c5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.300353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa118e5626f644b18b955a64e1ef34db, user_id=5bf30f9e6682405692158a3b7a29d0c5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.303812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.332183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1976/3995] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:00 2022] PUT /identity/v3/projects/aa118e5626f644b18b955a64e1ef34db/users/5bf30f9e6682405692158a3b7a29d0c5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:00.343429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.343787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.344133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.386035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] MFA Rules not processed for user `5bf30f9e6682405692158a3b7a29d0c5`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:00.388604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:00.434348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:00.506168 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2020/3996] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:00.510708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.622752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a080c5-7198-4f72-ab78-03580522d23a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.640625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.641259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.641539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.641811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.642451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.645880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.690967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1977/3997] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:00 2022] POST /identity/v3/projects => generated 411 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:00.692053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.706107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b8914-3e1f-4ef7-aa37-9f819ce20b4b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.806070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.806945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.807473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.807825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.808475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.810730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.846595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2021/3998] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:00 2022] POST /identity/v3/users => generated 312 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:00.851369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:00.889082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bafd-ad61-48d2-9895-0f6e322854aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:00.919738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:00.920585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:00.922424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:00.922938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:00.923760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:00.935347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:00.981334 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1978/3999] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:00.982355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.010961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bf897-6424-473d-87ca-8bb5b8c42371 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.037230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.038637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.039330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.039865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] PATH_INFO: `/v3/projects/2a6e62fbf8934656b63deacf53ea24c3/users/f7caa2d97b6e404e9eab07ed5fff89e5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.100244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6e62fbf8934656b63deacf53ea24c3, user_id=f7caa2d97b6e404e9eab07ed5fff89e5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.103379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7324fa7-1fac-4287-ae80-b30b4f5638f6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.139039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2022/4000] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:00 2022] PUT /identity/v3/projects/2a6e62fbf8934656b63deacf53ea24c3/users/f7caa2d97b6e404e9eab07ed5fff89e5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:01.143093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.156304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.156798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.156898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.161431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4f476-2772-4dae-98b2-d1947b2dfb75 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.228686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] MFA Rules not processed for user `f05599f233ca45c39ccd351c0432f1ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:01.237520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:01.349868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.351249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.351950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.353708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.354657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.361151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.385265 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1979/4001] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:01.402222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.423286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf2abe-3590-4c15-8bb9-be72751b085f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.429490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.430699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.431208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2023/4002] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:01.431672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.432353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] PATH_INFO: `/v3/projects/2a6e62fbf8934656b63deacf53ea24c3/users/f7caa2d97b6e404e9eab07ed5fff89e5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.435554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.436504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6e62fbf8934656b63deacf53ea24c3, user_id=f7caa2d97b6e404e9eab07ed5fff89e5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.436956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:01.441331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d3243-bdfd-4e06-9946-309bfff776ed admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.457590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1980/4003] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:01 2022] PUT /identity/v3/projects/2a6e62fbf8934656b63deacf53ea24c3/users/f7caa2d97b6e404e9eab07ed5fff89e5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:01.466786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef4465-0ede-4d47-addb-f7ce99a9bab7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.479010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.479010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.480344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.493557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:01.576077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] MFA Rules not processed for user `f7caa2d97b6e404e9eab07ed5fff89e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:01.578113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:01.581498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.583141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.583516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.583858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.585483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.589378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.606467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:01.698414 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2024/4004] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:01 2022] POST /identity/v3/projects => generated 405 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:01.713970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.754588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1981/4005] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3355 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:01.768317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.770551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:01.842653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842953cd-1601-487f-9477-28a586d32055 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.849532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.850724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.851244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.851936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.853610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.868198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.887679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b7cac-9556-49fb-a039-c1cf282eb836 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.902415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2025/4006] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:01 2022] POST /identity/v3/users => generated 303 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:01.906325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.919259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5bb0d-defd-4807-9ee9-283f631a2db9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.931419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.932438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.932939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.933431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.934293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:01.936144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:01.962017 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2026/4007] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:01.967340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:01.979823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a3111-12e4-4202-a094-6ad8c4e45a20 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:01.983487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.984317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.985547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.987265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.996077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:01.997241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:01.997839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:01.998647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:01.999693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:02.001543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:02.027222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2027/4008] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:02.040912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:02.063058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec3e5-2699-4453-8353-fdce3c6b2dd6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:02.106461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:02.111171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:02.111888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:02.112381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] PATH_INFO: `/v3/projects/d9836534a7794c98b32eaec0170e83b8/users/09443ab1a6ac4b5db4567fa891abe1d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:02.162368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9836534a7794c98b32eaec0170e83b8, user_id=09443ab1a6ac4b5db4567fa891abe1d1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:02.167747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52ddee-2a54-4dcf-8be9-7f1d8451a08f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:02.176695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:02.180536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:02.195928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1982/4009] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3355 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:02.203134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2028/4010] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:02 2022] PUT /identity/v3/projects/d9836534a7794c98b32eaec0170e83b8/users/09443ab1a6ac4b5db4567fa891abe1d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:07.421402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:07.443253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddee8f-c6c5-4f65-8684-2cecbba35e03 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:07.485350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:07.654576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:07.655231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:07.656766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:07.656766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] PATH_INFO: `/v3/users/5bf30f9e6682405692158a3b7a29d0c5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:07.657406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf30f9e6682405692158a3b7a29d0c5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:07.659753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:07.765343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf30f9e6682405692158a3b7a29d0c5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:07.766074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf30f9e6682405692158a3b7a29d0c5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:07.779966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf30f9e6682405692158a3b7a29d0c5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:07.796011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf30f9e6682405692158a3b7a29d0c5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:07.796786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf30f9e6682405692158a3b7a29d0c5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:07.837393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1983/4011] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:07 2022] DELETE /identity/v3/users/5bf30f9e6682405692158a3b7a29d0c5 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:07.841817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:07.842372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:07.842372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:07.879018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:07.880283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:07.987025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:08.034480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2029/4012] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:08.057642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:08.059715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:08.197884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390937be-832a-4dde-8a3b-4ad049d1792e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:08.205302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:08.205928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:08.206202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:08.206474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:08.230367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:08.231898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:08.269835 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1984/4013] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:09.118626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:09.121008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:09.149644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01354c68-2fa2-459d-85a1-fddd3e908e2e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:09.162425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:09.164470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:09.165007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:09.165647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] PATH_INFO: `/v3/projects/aa118e5626f644b18b955a64e1ef34db` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:09.167262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa118e5626f644b18b955a64e1ef34db)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:09.170152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:09.250178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa118e5626f644b18b955a64e1ef34db', 'request_id': 'req-a0a056fa-8200-4d80-99d4-606a7bc194ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.264220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa118e5626f644b18b955a64e1ef34db', 'request_id': 'req-a0a056fa-8200-4d80-99d4-606a7bc194ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.272481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa118e5626f644b18b955a64e1ef34db', 'request_id': 'req-a0a056fa-8200-4d80-99d4-606a7bc194ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.288248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] The token cache is being invalidate because project aa118e5626f644b18b955a64e1ef34db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:09.288734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.289896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.294787 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2030/4014] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:09 2022] DELETE /identity/v3/projects/aa118e5626f644b18b955a64e1ef34db => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:09.295848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:09.309190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f873126f-8d6a-4e88-86f6-06a2bd0c7e41 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:09.357130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:09.424389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:09.425082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:09.425433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:09.425785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] PATH_INFO: `/v3/users/f7caa2d97b6e404e9eab07ed5fff89e5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:09.427841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7caa2d97b6e404e9eab07ed5fff89e5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:09.434734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:09.528252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7caa2d97b6e404e9eab07ed5fff89e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.528938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7caa2d97b6e404e9eab07ed5fff89e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.541948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7caa2d97b6e404e9eab07ed5fff89e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.555299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7caa2d97b6e404e9eab07ed5fff89e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.558188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-93360419-cad0-4b4b-8945-97ca26476e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7caa2d97b6e404e9eab07ed5fff89e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:09.626523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1985/4015] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:09 2022] DELETE /identity/v3/users/f7caa2d97b6e404e9eab07ed5fff89e5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:09.877815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:09.879146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:09.909154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a056fa-8200-4d80-99d4-606a7bc194ec admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:10.064328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:10.065314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.066305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.066972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] PATH_INFO: `/v3/projects/2a6e62fbf8934656b63deacf53ea24c3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.068450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6e62fbf8934656b63deacf53ea24c3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:10.071504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:10.153539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6e62fbf8934656b63deacf53ea24c3', 'request_id': 'req-f40331fa-9075-4f1f-a060-99ac3581bf2b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:10.166435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6e62fbf8934656b63deacf53ea24c3', 'request_id': 'req-f40331fa-9075-4f1f-a060-99ac3581bf2b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:10.183149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6e62fbf8934656b63deacf53ea24c3', 'request_id': 'req-f40331fa-9075-4f1f-a060-99ac3581bf2b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:10.212173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] The token cache is being invalidate because project 2a6e62fbf8934656b63deacf53ea24c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:10.212640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:10.214544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f40331fa-9075-4f1f-a060-99ac3581bf2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:10.218573 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2031/4016] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:09 2022] DELETE /identity/v3/projects/2a6e62fbf8934656b63deacf53ea24c3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:10.224664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.224945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.225950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.269313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:10.270713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:10.400370 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1986/4017] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:10.408607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.408883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.409091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.421893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:10.446268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:10.448861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:10.476398 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2032/4018] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:10.480247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:10.492491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71de1ee-d98e-4a83-b3e3-e1b07f74ef16 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:10.511120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:10.512396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.512928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.513657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.514586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:10.515307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:10.517199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:10.538390 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1987/4019] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:05:10.542884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:10.563144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ae72a-d146-4da1-8dac-173f54422bf5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:10.575428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:10.576671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.577064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.577438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.579661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:10.582914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:10.685908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:10.686148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2033/4020] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:10 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:10.699911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98652-9f7d-4aef-a436-3e3d0e154b61 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:10.800879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:10.864808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:10.865674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.866070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.866446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.867214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:10.869254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:10.920672 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1988/4021] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:10 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:10.925352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:10.938942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e1eb-d916-402c-bd9b-794a18d25a9c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:10.944147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:10.944901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:10.945414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:10.945790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:10.946757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:10.948479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:10.973925 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2034/4022] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:10.975885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:10.989798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205496c9-1143-46db-9744-007e543392e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.011747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.012531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.012918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.013291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.014188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.015967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774632a7-34f0-4178-9add-1eb321f04c6f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.030332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1989/4023] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:11.034530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.050437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ec85d-9d4e-47da-824a-55fb4a81fdb4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.056942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.058025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.058424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.058840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] PATH_INFO: `/v3/projects/306f579ca5df4c3fa4d42221439425a7/users/34086609542f4d44bcb50c88141d1220/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.078600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=306f579ca5df4c3fa4d42221439425a7, user_id=34086609542f4d44bcb50c88141d1220, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.081447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.113326 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2035/4024] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:11 2022] PUT /identity/v3/projects/306f579ca5df4c3fa4d42221439425a7/users/34086609542f4d44bcb50c88141d1220/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:11.116881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.118116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.119186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.199544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] MFA Rules not processed for user `34086609542f4d44bcb50c88141d1220`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:11.200834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:11.385613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1990/4025] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:11.387979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.418449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.422211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:11.544112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaad8b9-24ce-49e7-8ae7-25b4d9ff8f44 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.562237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.563193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.563663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.564158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.564975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.567531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.600646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d04c7-ba80-4d63-b229-69f863524dbe None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.613061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.614929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.615328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.615911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] PATH_INFO: `/v3/users/5cb9430023d842b4a5d930e01f7340f3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.619548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb9430023d842b4a5d930e01f7340f3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.623281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.626901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2036/4026] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:11 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:11.632737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.704113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb9430023d842b4a5d930e01f7340f3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:11.704789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb9430023d842b4a5d930e01f7340f3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:11.713050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:11.726021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb9430023d842b4a5d930e01f7340f3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:11.742763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb9430023d842b4a5d930e01f7340f3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:11.743696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb9430023d842b4a5d930e01f7340f3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:11.764363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1991/4027] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:11 2022] DELETE /identity/v3/users/5cb9430023d842b4a5d930e01f7340f3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:11.772496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aae86-3a20-4d7b-8aed-edb05db410b5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.825802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:11.874896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.875873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.876962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.878340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.879098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.881362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.910102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2037/4028] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:11 2022] POST /identity/v3/users => generated 292 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:11.911489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.913619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:11.934932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b506c21-a614-4703-9d7c-d1467f3a20d3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.941212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.941934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.942285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.942836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.943562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.945752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:11.957449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1992/4029] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:11.961702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:11.977448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468483d-b3aa-4d8d-8a29-c357a61a44ae admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:11.982435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:11.983085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:11.983362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:11.983633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:11.984954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:11.986612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba3b63-e631-4a54-9d13-dabc72531ad2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.020021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2038/4030] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:12.027080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.066052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51db31b-4110-4210-bbd0-17020a0e94e5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.076201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.076798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.077077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.077375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] PATH_INFO: `/v3/projects/2cf31b4dea5d4b19b3e2d01f930427e6/users/1aae9d32275345d7ba6e857fc74a2686/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.105962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf31b4dea5d4b19b3e2d01f930427e6, user_id=1aae9d32275345d7ba6e857fc74a2686, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.110315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.139840 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1993/4031] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:12 2022] PUT /identity/v3/projects/2cf31b4dea5d4b19b3e2d01f930427e6/users/1aae9d32275345d7ba6e857fc74a2686/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:12.151958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.153099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.153922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.212536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] MFA Rules not processed for user `1aae9d32275345d7ba6e857fc74a2686`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:12.212536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:12.282060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.284955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:12.360312 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2039/4032] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:12.365293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.440202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53c511-d4bb-4e10-93d1-eeb489c34138 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.454065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.454755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.458076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.465006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] PATH_INFO: `/v3/projects/3a9bed43edcf4c5594ffde33282bc48c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.470358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9bed43edcf4c5594ffde33282bc48c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.477645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.528663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c38451-f233-49c2-9ea7-c30a108cf333 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.536779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.539038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.539762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.540257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.541393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.549362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.592839 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2040/4033] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:12.594111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.609639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9bed43edcf4c5594ffde33282bc48c', 'request_id': 'req-a0227caf-e8db-4fa2-99f0-4bdab2e217be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:12.628555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9bed43edcf4c5594ffde33282bc48c', 'request_id': 'req-a0227caf-e8db-4fa2-99f0-4bdab2e217be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:12.639343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:12.663000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9bed43edcf4c5594ffde33282bc48c', 'request_id': 'req-a0227caf-e8db-4fa2-99f0-4bdab2e217be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:12.692665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] The token cache is being invalidate because project 3a9bed43edcf4c5594ffde33282bc48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:12.692954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:12.696424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a0227caf-e8db-4fa2-99f0-4bdab2e217be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:12.702189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27a14b-a52c-4afc-81a0-2154c7cefce4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.706613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1994/4034] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:12 2022] DELETE /identity/v3/projects/3a9bed43edcf4c5594ffde33282bc48c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:12.709808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4dcb-1e34-44f9-b6a6-70009aae804d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.710147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4dcb-1e34-44f9-b6a6-70009aae804d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.710422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f4dcb-1e34-44f9-b6a6-70009aae804d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.720510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.721063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.721552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.721672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.722234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.724190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.741786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0f3f4dcb-1e34-44f9-b6a6-70009aae804d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:12.743612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0f3f4dcb-1e34-44f9-b6a6-70009aae804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:12.753284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1995/4035] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:12.762048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.762395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.762772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.767847 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2041/4036] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:12.771397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.774537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:12.788687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:12.790049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:12.790857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0631-29b5-494a-b762-704b8d612329 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.795731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.796332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.796605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.796871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.797784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.799122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.802613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1996/4037] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:12.804842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.812241 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2042/4038] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:12.825849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.831222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a557e69-5513-4b06-9cc7-4fb840760f51 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.842922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b1d62-3593-4c4e-9f04-c62bdeada7aa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.848610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.848610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.848610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.848610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.849070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:12.849070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.849438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.851106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.851536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.851918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] PATH_INFO: `/v3/projects/d56ba00623954a158eda25112ea04d6d/users/7d060d99930f4922b358da724a9303cb/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.852166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.860997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:12.877639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56ba00623954a158eda25112ea04d6d, user_id=7d060d99930f4922b358da724a9303cb, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.880894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:12.893457 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1997/4039] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:12.898047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.901310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2043/4040] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:12 2022] PUT /identity/v3/projects/d56ba00623954a158eda25112ea04d6d/users/7d060d99930f4922b358da724a9303cb/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:12.905191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:12.910207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafd3ee-d2f9-46d1-a5ca-0bbfc21c2a01 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:12.984009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:12.984834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:12.985268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:12.985444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:12.986579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:12.988631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.004612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844601f5-0fae-4841-a144-65421e3aea74 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.010768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.011384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.011909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.012186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.013083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.015451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.029248 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2044/4041] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:13.033224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.035224 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1998/4042] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:12 2022] POST /identity/v3/projects => generated 363 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:13.040352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.046029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5dc9d-c20a-476c-aa75-9d7bad799e8d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.134224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.134970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.135251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.135527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] PATH_INFO: `/v3/projects/d56ba00623954a158eda25112ea04d6d/users/7d060d99930f4922b358da724a9303cb/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.138289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56ba00623954a158eda25112ea04d6d, user_id=7d060d99930f4922b358da724a9303cb, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.141190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00122d-8959-4a8f-9696-59a52e0dea34 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.158550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2045/4043] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:13 2022] PUT /identity/v3/projects/d56ba00623954a158eda25112ea04d6d/users/7d060d99930f4922b358da724a9303cb/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:13.160487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d541-04c7-43cd-8ec0-186825d0765f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.171615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.173811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.174001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.210349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:13.230274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] MFA Rules not processed for user `7d060d99930f4922b358da724a9303cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:13.231668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:13.314548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.315345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.315617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.315895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.316537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.323402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.355072 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 1999/4044] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:13 2022] POST /identity/v3/users => generated 282 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:13.359269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.362773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2046/4045] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:13.376472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48139bf0-0aa9-4f09-98ba-d4257172daa9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.383893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.389827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:13.391281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.394302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.394687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.394975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.395639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.399392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.424485 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2000/4046] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:13.425707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.437870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e6ec6-8d16-4e4e-973e-ed6b6ea0b6b8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.449399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.449966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.450171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.450768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.451115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.452817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.461651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:13.483551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2001/4047] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:13.487299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.511746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790e22-6544-4578-a9cf-e07dea96afb9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.523644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8797c-6a1a-4ad3-b525-b7f2298d8fd4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:13.527657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.528350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.528632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.528907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] PATH_INFO: `/v3/projects/0659f11e49b546499d8efc9fb6c4a615/users/fde169e8804e47e68b2610702a596807/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.529525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:13.530253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.531093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.531093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.533835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:13.550681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0659f11e49b546499d8efc9fb6c4a615, user_id=fde169e8804e47e68b2610702a596807, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.554300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff2cc6-4cc9-41c7-9ade-50a021bf3eb8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.589345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2002/4048] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:13 2022] PUT /identity/v3/projects/0659f11e49b546499d8efc9fb6c4a615/users/fde169e8804e47e68b2610702a596807/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:13.601185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f07e-7c40-4dab-bd47-30e549284c77 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:13.601687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f07e-7c40-4dab-bd47-30e549284c77 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:13.602063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357f07e-7c40-4dab-bd47-30e549284c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:13.640540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:13.657412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:13.663928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9357f07e-7c40-4dab-bd47-30e549284c77 None None] MFA Rules not processed for user `fde169e8804e47e68b2610702a596807`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:13.666462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9357f07e-7c40-4dab-bd47-30e549284c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:13.689595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2047/4049] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:13.767238 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2003/4050] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:13.782861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:13.784894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:13.894935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:13.939602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dc6cf-866e-477a-a696-ea661aa5aa95 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:14.132737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:14.133372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:14.133653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:14.133924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:14.156855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:14.159138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:14.193541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2048/4051] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:14.721864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34319-037d-49a6-ba64-cd26aa084058 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:14.722409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34319-037d-49a6-ba64-cd26aa084058 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:14.722409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34319-037d-49a6-ba64-cd26aa084058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:14.821815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:14.835374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4fc34319-037d-49a6-ba64-cd26aa084058 None None] MFA Rules not processed for user `09443ab1a6ac4b5db4567fa891abe1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:14.836940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4fc34319-037d-49a6-ba64-cd26aa084058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:14.977977 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2004/4052] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:14.979687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a35-fdf1-4886-9232-95d4991b531f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:14.979825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a35-fdf1-4886-9232-95d4991b531f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:14.980058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a35-fdf1-4886-9232-95d4991b531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:14.992554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b9b0-126d-4d94-9a9a-cc66d4eface3 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:14.999891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:15.000681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:15.001104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:15.002687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] PATH_INFO: `/v3/users/fde169e8804e47e68b2610702a596807` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:15.003330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde169e8804e47e68b2610702a596807)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:15.006341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:15.041814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d9092a35-fdf1-4886-9232-95d4991b531f None None] MFA Rules not processed for user `f05599f233ca45c39ccd351c0432f1ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:15.044819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9092a35-fdf1-4886-9232-95d4991b531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:15.123661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde169e8804e47e68b2610702a596807'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:15.125877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde169e8804e47e68b2610702a596807'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:15.137881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde169e8804e47e68b2610702a596807'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:15.167787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde169e8804e47e68b2610702a596807'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:15.169127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde169e8804e47e68b2610702a596807'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:15.170108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:15.178153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:15.214269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2005/4053] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:15.232370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:15.232713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:15.232999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:15.239391 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2049/4054] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:14 2022] DELETE /identity/v3/users/fde169e8804e47e68b2610702a596807 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:15.240892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:15.244643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:15.313523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:15.317830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:15.460706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2afa-8f51-44e1-960d-f2804ccf0cba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:15.469682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:15.470393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:15.470713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:15.470992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:15.475053 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2006/4055] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:15.478434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:15.489383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:15.492595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:15.646341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:15.649848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe61d1-cdf7-433e-89b2-8a5938059cdd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:15.651243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:15.663196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:15.665384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:15.666798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:15.668301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:15.685723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2050/4056] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:15 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:15.691649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:15.693522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87421dfb-01f4-4ece-ad37-aba7d90ce913 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:15.722339 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2007/4057] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:16.616334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:16.617909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:16.635514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77918a2e-6968-4df3-9d3c-fdc69bd843ee service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:16.644382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:16.645874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:16.646258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:16.647418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] PATH_INFO: `/v3/projects/0659f11e49b546499d8efc9fb6c4a615` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:16.648954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0659f11e49b546499d8efc9fb6c4a615)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:16.651997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:16.817983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0659f11e49b546499d8efc9fb6c4a615', 'request_id': 'req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:16.832114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0659f11e49b546499d8efc9fb6c4a615', 'request_id': 'req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:16.854867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0659f11e49b546499d8efc9fb6c4a615', 'request_id': 'req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:16.887690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] The token cache is being invalidate because project 0659f11e49b546499d8efc9fb6c4a615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:16.889518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:16.890393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:16.901172 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2051/4058] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:16 2022] DELETE /identity/v3/projects/0659f11e49b546499d8efc9fb6c4a615 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:16.901960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767ed7d-daec-4719-ae40-643c69900c47 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:16.902416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767ed7d-daec-4719-ae40-643c69900c47 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:16.902581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767ed7d-daec-4719-ae40-643c69900c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:16.985241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3767ed7d-daec-4719-ae40-643c69900c47 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:16.986616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3767ed7d-daec-4719-ae40-643c69900c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:17.000558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.002761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:17.058426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:17.281365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2008/4059] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:17.292046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.292382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.292791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.331124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ad6ed-593e-4f9c-bfc5-ae06639c79f4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:17.359125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:17.359923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.359923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.360950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.361948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:17.370851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:17.371351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:17.426443 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2009/4060] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:17.427639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.452990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:17.454814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:17.470455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358cad9-f612-45b7-b68d-ee10b8c191d4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:17.497859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:17.498629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.499006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.499378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.500939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:17.501107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2052/4061] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:05:17.501476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:17.504744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:17.538938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2010/4062] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:17.543301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.589266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aee00-adc4-4c8a-a4e2-58beb55d2a86 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:17.610040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:17.610692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.611127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.611406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.612043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:17.620380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:17.709019 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2053/4063] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:17 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:17.711455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.729766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6befa-383d-49c2-817f-5db18b76f91c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:17.761323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:17.843624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:17.844654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.845123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.845498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.846338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:17.851451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:17.914534 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2011/4064] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:17 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:17.919291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.937872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a50c82-83b8-4e61-999f-0ac681e28c81 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:17.944578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:17.945387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:17.945663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:17.945990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:17.947241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:17.948861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:17.973563 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2054/4065] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:17.976663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:17.996653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e22bc6-fb50-412a-97ce-792fb206e5b7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:18.008898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:18.010339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.010606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.011037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.011671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:18.013697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de69a1-f072-4ee3-a567-20ab69ef92da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:18.043795 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2012/4066] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:18.047593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:18.062262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f644e44-1049-4c54-87d1-80b0cb85ecb1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:18.074741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:18.075424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.075704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.075988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] PATH_INFO: `/v3/projects/00556c830a5b42da97d546db3eb1cb41/users/1cf3e6295def43af95fb77eb1b15e6e9/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.132126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00556c830a5b42da97d546db3eb1cb41, user_id=1cf3e6295def43af95fb77eb1b15e6e9, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:18.136627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:18.169621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2055/4067] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:18 2022] PUT /identity/v3/projects/00556c830a5b42da97d546db3eb1cb41/users/1cf3e6295def43af95fb77eb1b15e6e9/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:18.180803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.181091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.181545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.275108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] MFA Rules not processed for user `1cf3e6295def43af95fb77eb1b15e6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:18.276586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:18.458750 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2013/4068] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:18.459368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:18.605103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9896-af65-4169-92df-0ad657750d7a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:18.618644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:18.619315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.619688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.620056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.620791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:18.622841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:18.703492 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2056/4069] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:18 2022] POST /identity/v3/projects => generated 383 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:18.707328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:18.735489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05058328-7e6c-42c3-97ba-12b02cfcb463 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:18.883210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:18.884620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.885006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.885194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.885934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:18.889650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:18.940312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:18.952840 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2014/4070] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:18 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:18.957537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb489151-be97-4df0-8a75-c28364c8711b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:18.964763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:18.965787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:18.966309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:18.967321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:18.969623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:18.971605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:18.989409 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2057/4071] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:18.991578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:19.010768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a2149-d8a0-47a2-abe5-c538a4fa43a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:19.038055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:19.039107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:19.039657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:19.040199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] PATH_INFO: `/v3/projects/987cf28985e64e3db6b009cc36863dd3/users/60f8862c51d0435f832497fd3819ec83/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:19.055719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:19.058159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:19.133960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251ad91-488f-442f-bf0d-1411b8d64886 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:19.139258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=987cf28985e64e3db6b009cc36863dd3, user_id=60f8862c51d0435f832497fd3819ec83, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:19.149111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:19.154429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:19.160314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:19.160965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:19.161491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:19.161942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] PATH_INFO: `/v3/users/dad0b98fdcf145a1960121573524940d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:19.164925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad0b98fdcf145a1960121573524940d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:19.169465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:19.175868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2015/4072] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:18 2022] PUT /identity/v3/projects/987cf28985e64e3db6b009cc36863dd3/users/60f8862c51d0435f832497fd3819ec83/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:19.177117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:19.205010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:19.284239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad0b98fdcf145a1960121573524940d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:19.286538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad0b98fdcf145a1960121573524940d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:19.306104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad0b98fdcf145a1960121573524940d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:19.338041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f8902-fad7-4027-9d72-087db3c57aca admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:19.346916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad0b98fdcf145a1960121573524940d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:19.351219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad0b98fdcf145a1960121573524940d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:19.372054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:19.372777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:19.372998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:19.373269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:19.373913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:19.379672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c6232-702c-4d2e-969a-8e665b1bfcce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:19.397589 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2058/4073] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:19 2022] DELETE /identity/v3/users/dad0b98fdcf145a1960121573524940d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:19.421004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:19.431265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:19.450027 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2016/4074] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:19.648147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d7c31-40e8-429d-bcb2-15fefd50967e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:19.662862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:19.664404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:19.664836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:19.665826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] PATH_INFO: `/v3/projects/987cf28985e64e3db6b009cc36863dd3/users/60f8862c51d0435f832497fd3819ec83/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:19.671878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] RBAC: Authorizing `identity:create_grant(project_id=987cf28985e64e3db6b009cc36863dd3, user_id=60f8862c51d0435f832497fd3819ec83, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:19.675141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:19.708604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2059/4075] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:19 2022] PUT /identity/v3/projects/987cf28985e64e3db6b009cc36863dd3/users/60f8862c51d0435f832497fd3819ec83/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:19.721050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:19.721418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:19.721955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:19.807763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] MFA Rules not processed for user `60f8862c51d0435f832497fd3819ec83`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:19.808372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:19.960467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:19.971981 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2017/4076] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:19.983500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60b9b4-3abd-4921-8ad1-6a90bd69892f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:19.997999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.000999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.002644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.003867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.007193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.011770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invalid input for field 'name': '' is too short Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: On instance['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: On instance['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: '' Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise error Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application '' Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.049087 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:05:20.051075 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application '' Sep 24 21:05:20.052417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:05:20.063719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.080444 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2060/4077] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:05:19 2022] POST /identity/v3/domains => generated 283 bytes in 116 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:05:20.081649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485cdd-c5b8-4995-a6d3-9607830a7bd8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:20.090634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.091590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.092051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.092529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.094305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.096773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: On instance['name']: Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: On instance['name']: Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise error Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.113397 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post Sep 24 21:05:20.115135 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:05:20.117305 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.150219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.160757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2018/4078] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:05:20 2022] POST /identity/v3/domains => generated 412 bytes in 96 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:05:20.166334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9fa31-6d5e-453c-a0d9-2e57e443d5da tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:20.171732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.172781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.173386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.173969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.175284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.178509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.208384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2061/4079] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:05:20 2022] POST /identity/v3/domains => generated 277 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:20.209953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.228842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d08681-1a72-43b0-8925-88700ec75ffb tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:20.389083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.389728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.399017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.399722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/c293bb17450e43b6ac2aada1141fd178` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.438076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c293bb17450e43b6ac2aada1141fd178)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.440283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.448889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete Sep 24 21:05:20.498427 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. Sep 24 21:05:20.499876 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:20.500855 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2019/4080] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:05:20 2022] DELETE /identity/v3/domains/c293bb17450e43b6ac2aada1141fd178 => generated 120 bytes in 291 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:05:20.502314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.504601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:20.507940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.564456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4897d-0b25-47ae-9417-0290668cc0bf tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:20.581432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.582106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.582669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.582946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/c293bb17450e43b6ac2aada1141fd178` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.587509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c293bb17450e43b6ac2aada1141fd178)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.591877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.661614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:20.665585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:20.758037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:20.765707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:20.772978 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2020/4081] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:05:20 2022] PATCH /identity/v3/domains/c293bb17450e43b6ac2aada1141fd178 => generated 278 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:20.779436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9835-8d02-4d59-9cd3-195977d38297 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:20.788698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:20.792318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:20.820371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:20.965890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:20.966928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:20.968541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:20.969012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] PATH_INFO: `/v3/projects/6b7f252278c74e2d84de081cd3d5a69e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:20.971469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7f252278c74e2d84de081cd3d5a69e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:20.980443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:20.992802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe7523-3d2f-4702-af83-84aa8184496a tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.007285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.008122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.008853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.009524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/c293bb17450e43b6ac2aada1141fd178` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.035679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c293bb17450e43b6ac2aada1141fd178)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.039049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.086051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.098108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7f252278c74e2d84de081cd3d5a69e', 'request_id': 'req-0712caa5-54c2-46c8-be0c-cb9565db5d67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.108362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7f252278c74e2d84de081cd3d5a69e', 'request_id': 'req-0712caa5-54c2-46c8-be0c-cb9565db5d67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.116970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7f252278c74e2d84de081cd3d5a69e', 'request_id': 'req-0712caa5-54c2-46c8-be0c-cb9565db5d67'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.135491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] The token cache is being invalidate because project 6b7f252278c74e2d84de081cd3d5a69e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:21.136207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.137433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0712caa5-54c2-46c8-be0c-cb9565db5d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.141094 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2062/4082] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:20 2022] DELETE /identity/v3/projects/6b7f252278c74e2d84de081cd3d5a69e => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:21.147236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b19-198b-4931-863d-9595a97a3611 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.147403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.147700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b19-198b-4931-863d-9595a97a3611 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.150083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4058b19-198b-4931-863d-9595a97a3611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.199901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b4058b19-198b-4931-863d-9595a97a3611 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:21.202598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4058b19-198b-4931-863d-9595a97a3611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:21.263618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.277989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.297781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.325423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] The token cache is being invalidate because project c293bb17450e43b6ac2aada1141fd178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:21.328779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.330720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.348701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:21.354351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.354921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.355747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2063/4083] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:21.360057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c293bb17450e43b6ac2aada1141fd178', 'request_id': 'req-67c360a9-8d35-49be-8d0a-480bce4bd548'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.362916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.363871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.363871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.382863 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2021/4084] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:05:20 2022] DELETE /identity/v3/domains/c293bb17450e43b6ac2aada1141fd178 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:21.388687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.392032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:21.415123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:21.416385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:21.528568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:21.535940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c360a9-8d35-49be-8d0a-480bce4bd548 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.544344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.545073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.546268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.546685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/c85ec08a82cc4c31bd36a9da4e52a2da` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.563351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Could not find domain: c85ec08a82cc4c31bd36a9da4e52a2da.: keystone.exception.DomainNotFound: Could not find domain: c85ec08a82cc4c31bd36a9da4e52a2da. Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c85ec08a82cc4c31bd36a9da4e52a2da. Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:21.570994 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:21.572367 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:05:21.573647 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c85ec08a82cc4c31bd36a9da4e52a2da. Sep 24 21:05:21.574929 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.581709 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2022/4085] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:05:21 2022] DELETE /identity/v3/domains/c85ec08a82cc4c31bd36a9da4e52a2da => generated 112 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:05:21.582802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.597282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f67a-0338-48dd-b664-bfd497279be4 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.604684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2064/4086] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:21.606042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.606765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.607050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.607333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.608115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.609632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.609957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.623482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7d89b-e21e-4657-9cbf-838bde8096d0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.634686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2023/4087] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/domains => generated 253 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:21.638826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.649359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.650388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.652479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.653084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.654161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:21.654713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.656885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.675844 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2065/4088] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:21.683136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.712367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b6a8-fe53-4d21-af79-c8b6c8774d1f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.755777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b683d74-1878-4893-9969-0cf48af85f5f tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.765386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.766000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.766337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.766650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.767381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.769421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.788706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.sql.core [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1083702413' for key 'project.ixu_project_name_domain_id'") Sep 24 21:05:21.788706 ncdv-master-1 devstack@keystone.service[99133]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 21:05:21.788706 ncdv-master-1 devstack@keystone.service[99133]: [parameters: {'id': 'd6b0f3a5f57043539a07203e65a59fc6', 'name': 'tempest-domain-dup-1083702413', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 24 21:05:21.788706 ncdv-master-1 devstack@keystone.service[99133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1083702413.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1083702413. Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 21:05:21.802320 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1083702413' for key 'project.ixu_project_name_domain_id'") Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 21:05:21.803886 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application yield resource Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise self.value Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application session.commit() Sep 24 21:05:21.805356 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.flush() Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application compat.raise_( Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 21:05:21.806877 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 21:05:21.808243 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 21:05:21.809523 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1083702413' for key 'project.ixu_project_name_domain_id'") Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [parameters: {'id': 'd6b0f3a5f57043539a07203e65a59fc6', 'name': 'tempest-domain-dup-1083702413', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.810887 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1083702413. Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:21.813980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1083702413. Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:21.815535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2024/4089] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/domains => generated 212 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:05:21.816778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.826308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71d116-0e3c-4fdc-a28b-6633deca2af7 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:21.833858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.835251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.835527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.835800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/505135b6974c4d129d58fbf0f363c792` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.859245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=505135b6974c4d129d58fbf0f363c792)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.863268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:21.863769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.865205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:21.866326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:21.866833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:21.868417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:21.884082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:21.931839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.935314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.981307 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2066/4090] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/projects => generated 391 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:21.981534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:21.987716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:21.988935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.992062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:21.997403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2025/4091] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:05:21 2022] PATCH /identity/v3/domains/505135b6974c4d129d58fbf0f363c792 => generated 254 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:22.001754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.002765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.069475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:22.149755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a23e9-94ca-4367-b721-a02764934bcf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:22.156504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:22.157126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.157406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.159121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.159121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:22.163458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:22.165283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895f9cf-0aa7-4aa9-b38e-126b7cef26e3 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:22.228033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2067/4092] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:21 2022] POST /identity/v3/users => generated 296 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:22.234813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.236947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.276817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82813bd4-f525-4005-94ab-54416ac465e8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:22.296017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:22.297256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.297546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.300107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.300874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:22.302560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c3572-e546-4358-b946-a107cce2bc68 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:22.304958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:22.305706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.307005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.307565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/505135b6974c4d129d58fbf0f363c792` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.342859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2068/4093] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:22.344954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.345272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=505135b6974c4d129d58fbf0f363c792)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:22.345888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.346393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.347437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:22.384769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.395109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:22.397202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.409270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '505135b6974c4d129d58fbf0f363c792'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.476086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:22.495747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.511098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.534636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.570123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] The token cache is being invalidate because project 505135b6974c4d129d58fbf0f363c792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:22.570583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.571480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.598197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2069/4094] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:22.601088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.606120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.610604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.611550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.615954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '505135b6974c4d129d58fbf0f363c792', 'request_id': 'req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:22.633508 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2026/4095] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:05:22 2022] DELETE /identity/v3/domains/505135b6974c4d129d58fbf0f363c792 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:22.645597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.650112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d838c-2b70-4d10-aec2-1272096eb7bc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:22.653864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.675195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:22.869855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:22.870477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.870795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.871056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.875054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:22.881255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:22.884690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0042-e4d9-4ba4-8409-c3de56ee44c2 tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:22.913845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f8862c51d0435f832497fd3819ec83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '987cf28985e64e3db6b009cc36863dd3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:22.915112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:22.917748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:22.918049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] PATH_INFO: `/v3/domains/c293bb17450e43b6ac2aada1141fd178` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:22.923931 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2070/4096] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:22.925039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.934256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:22.939946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Could not find domain: c293bb17450e43b6ac2aada1141fd178.: keystone.exception.DomainNotFound: Could not find domain: c293bb17450e43b6ac2aada1141fd178. Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c293bb17450e43b6ac2aada1141fd178. Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:22.951913 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:22.953176 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:05:22.954282 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c293bb17450e43b6ac2aada1141fd178. Sep 24 21:05:22.956517 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:22.956517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2027/4097] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:05:22 2022] PATCH /identity/v3/domains/c293bb17450e43b6ac2aada1141fd178 => generated 112 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:05:22.956517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:22.961236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.004910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99017474-e996-4004-9a56-81d936963c9d tempest-DomainsNegativeTestJSON-1649791026 tempest-DomainsNegativeTestJSON-1649791026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.025064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.025759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.026045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.026325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] PATH_INFO: `/v3/projects/648cf4f432a3400b8dcf2b57a99d082e/users/762f4e35de9e4083bd0e7a4d0de3c136/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.075327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=648cf4f432a3400b8dcf2b57a99d082e, user_id=762f4e35de9e4083bd0e7a4d0de3c136, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.079156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.103252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9f3ee-cf72-48e4-8585-4c15371f1b14 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.111975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2028/4098] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:22 2022] PUT /identity/v3/projects/648cf4f432a3400b8dcf2b57a99d082e/users/762f4e35de9e4083bd0e7a4d0de3c136/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:23.115896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:23.131093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.231241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c00cb8-99ac-48f8-8129-606f091f3c7d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.236509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.237056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.237381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.237656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.239156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.239805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.241679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.242184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.242768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] PATH_INFO: `/v3/users/1cf3e6295def43af95fb77eb1b15e6e9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.244762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf3e6295def43af95fb77eb1b15e6e9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.247381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.257800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.259377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35d318-99ad-47d1-ab57-50817af5c6eb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.288527 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2071/4099] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:23.290608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.291234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.291791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.334378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf3e6295def43af95fb77eb1b15e6e9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:23.335539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf3e6295def43af95fb77eb1b15e6e9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:23.335980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:23.343563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.348880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf3e6295def43af95fb77eb1b15e6e9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:23.363725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2072/4100] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:23.367108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:23.370552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf3e6295def43af95fb77eb1b15e6e9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:23.371567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf3e6295def43af95fb77eb1b15e6e9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:23.373235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.397152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb9267-e871-453b-9335-5f23d5f124a8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.398679 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2029/4101] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:23 2022] DELETE /identity/v3/users/1cf3e6295def43af95fb77eb1b15e6e9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:23.399705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:23.405214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:23.477197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:23.543069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86c124-1e39-4b7c-917d-f9485d60ad1d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.549896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.550530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.550840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.551082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.557506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.558491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.559038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.559531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.569008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.585297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.588377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73afdf2-3821-47b8-9207-29ebdab14d48 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.599122 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2073/4102] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:23.601740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.602088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.602425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.623286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:23.624702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.681766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.683545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.698430 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2030/4103] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:23.711156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2074/4104] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:23.739170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:23.742931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:23.757797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e50931-2a9e-43be-8dc6-d356bdb24488 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:23.763120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:23.764538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:23.765192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:23.765637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:23.783613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:23.787984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:23.802218 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2031/4105] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:24.894872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:24.896405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:24.920499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fea2-8eec-4898-98cf-b642ba3435e2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:24.938663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:24.939600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:24.940840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:24.941752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] PATH_INFO: `/v3/projects/00556c830a5b42da97d546db3eb1cb41` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:24.946801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00556c830a5b42da97d546db3eb1cb41)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:24.951705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:25.027488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00556c830a5b42da97d546db3eb1cb41', 'request_id': 'req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.036799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00556c830a5b42da97d546db3eb1cb41', 'request_id': 'req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.046429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00556c830a5b42da97d546db3eb1cb41', 'request_id': 'req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.064648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] The token cache is being invalidate because project 00556c830a5b42da97d546db3eb1cb41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:25.064942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.065675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.071276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2075/4106] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:24 2022] DELETE /identity/v3/projects/00556c830a5b42da97d546db3eb1cb41 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:25.073071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:25.086330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c62f5a-f9c9-47a5-941b-b58b8d396aa7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:25.231605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:25.232282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:25.232565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:25.233080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] PATH_INFO: `/v3/users/60f8862c51d0435f832497fd3819ec83` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:25.235603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f8862c51d0435f832497fd3819ec83)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:25.239414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:25.349321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:25.367520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f8862c51d0435f832497fd3819ec83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.368522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f8862c51d0435f832497fd3819ec83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.391987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f8862c51d0435f832497fd3819ec83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.424060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f8862c51d0435f832497fd3819ec83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.426940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f8862c51d0435f832497fd3819ec83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:25.527892 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2032/4107] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:25 2022] DELETE /identity/v3/users/60f8862c51d0435f832497fd3819ec83 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:26.530144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:26.532244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:26.734310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:26.800935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:26.803844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:26.820781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f7b1d-ddbc-47ce-b294-375a89089ab6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:27.085951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0fc4-ab6e-427c-98b3-4816f079a97b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:27.105399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:27.106063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.106346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.106655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] PATH_INFO: `/v3/projects/987cf28985e64e3db6b009cc36863dd3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.109466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01312219-8751-4ce1-9089-976893c06646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:27.110103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01312219-8751-4ce1-9089-976893c06646 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.110384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01312219-8751-4ce1-9089-976893c06646 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.110690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01312219-8751-4ce1-9089-976893c06646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.113281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=987cf28985e64e3db6b009cc36863dd3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:27.117320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:27.133885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01312219-8751-4ce1-9089-976893c06646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:27.151241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01312219-8751-4ce1-9089-976893c06646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:27.153472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01312219-8751-4ce1-9089-976893c06646 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:27.187831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2076/4108] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:27.268373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987cf28985e64e3db6b009cc36863dd3', 'request_id': 'req-e5082a10-669c-46b0-948a-e2f86483a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:27.293446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987cf28985e64e3db6b009cc36863dd3', 'request_id': 'req-e5082a10-669c-46b0-948a-e2f86483a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:27.323702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987cf28985e64e3db6b009cc36863dd3', 'request_id': 'req-e5082a10-669c-46b0-948a-e2f86483a842'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:27.341268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] The token cache is being invalidate because project 987cf28985e64e3db6b009cc36863dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:27.341868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:27.344329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e5082a10-669c-46b0-948a-e2f86483a842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:27.353137 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2033/4109] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:26 2022] DELETE /identity/v3/projects/987cf28985e64e3db6b009cc36863dd3 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:27.355605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.357177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.357466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.411240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:27.413496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:27.598799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2077/4110] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:27.624439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.624439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.624801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.660300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:27.661749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:27.673904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2034/4111] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:27.678474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:27.694281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6525a-364e-4b1c-94d0-12c907445427 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:27.700052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:27.700807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.701183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.702106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.702901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:27.703302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:27.705078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:27.719864 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2078/4112] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:27.723999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:27.734764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3882054-1029-4288-ade2-98e50f0bacc9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:27.740194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:27.741330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.741611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.741883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.742548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:27.744489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:27.794226 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2035/4113] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:27 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:27.798166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:27.813132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4089b-2289-4822-bbab-d322c0af41e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:27.950956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:27.951747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:27.952030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:27.952303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:27.953545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:27.956726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:27.971670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:27.995316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.020070 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2079/4114] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:27 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:28.025593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9a38c-7b3b-4999-9e34-6b3a5fbc5702 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.041505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.042001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.044117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.044896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.045824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.049520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.067712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2036/4115] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:28.074528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.087639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bca90-ffee-4171-b124-60606c319fda admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.101374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.102003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.102282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.102591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.103510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.105323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2322a4-25ff-42c4-93f7-85a882f44ed7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.127827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.132112 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2080/4116] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:28.144431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92726b10-27bf-4248-b5a4-8f48ef47e9ff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.160054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.160983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.161440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.162236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] PATH_INFO: `/v3/projects/e06cf38e47a34bb38d83c91e2670c2aa/users/63f83c27e4fa4658ab097d5de57ab6cb/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.173133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:28.188656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06cf38e47a34bb38d83c91e2670c2aa, user_id=63f83c27e4fa4658ab097d5de57ab6cb, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.192675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.230301 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2037/4117] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:28 2022] PUT /identity/v3/projects/e06cf38e47a34bb38d83c91e2670c2aa/users/63f83c27e4fa4658ab097d5de57ab6cb/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:28.232384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.232913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.233148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.279373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] MFA Rules not processed for user `63f83c27e4fa4658ab097d5de57ab6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:28.281114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:28.401770 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2081/4118] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:28.405845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.564464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f77804-aa99-40dc-bc8d-50e13252d97c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.576137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.576985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.577271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.578122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.578873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.582413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.633463 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2038/4119] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:28 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:28.633872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.661674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2381e78-a6a3-4a5c-9405-1c6b6b11e583 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.830971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.831674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.832023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.832365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.833096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.841321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.872236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2082/4120] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:28 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:28.876436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:28.919638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2601afc-cacd-4c4b-a0d8-2c362a630e71 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:28.937837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:28.938436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:28.938791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:28.943315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:28.951471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:28.953879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:28.981620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2039/4121] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:28.985655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.002539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba0a3-64c3-48b4-b0be-d80aa755a403 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.012845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.013529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.013809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.014086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] PATH_INFO: `/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39/users/6c2c02fedb1343c187c72f0c572b3a3f/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.043188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a8c1f0c0d6431d858377a0ea0adf39, user_id=6c2c02fedb1343c187c72f0c572b3a3f, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.046611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.073401 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2083/4122] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:28 2022] PUT /identity/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39/users/6c2c02fedb1343c187c72f0c572b3a3f/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:29.077414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.090770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcf569-8d8c-4ca3-9c84-fdf7c3668d8d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.176109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.177458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.177458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.178203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.179253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.180888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2687-97f2-40ce-8ce2-bbec53d36134 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.206953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2040/4123] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:29.211607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.228421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8820c4-85d3-4832-a713-1f34f7cc0e03 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.234362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.235367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.235889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.236190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] PATH_INFO: `/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39/users/6c2c02fedb1343c187c72f0c572b3a3f/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.238861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a8c1f0c0d6431d858377a0ea0adf39, user_id=6c2c02fedb1343c187c72f0c572b3a3f, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.242151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.254535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2084/4124] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:29 2022] PUT /identity/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39/users/6c2c02fedb1343c187c72f0c572b3a3f/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:29.266247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.266549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.266948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.308142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] MFA Rules not processed for user `6c2c02fedb1343c187c72f0c572b3a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:29.311297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:29.388310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:29.461191 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2041/4125] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:29.464925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.483567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37e259-cc2e-459f-949b-e00faa9c287f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.490700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.492503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.492964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.493421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.494242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.496547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.553963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.556496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:29.596152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2085/4126] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:29 2022] POST /identity/v3/projects => generated 334 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:29.599477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.611187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:29.696591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7779213-21b6-4e7a-9c13-e09f6cf6cfdf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.718604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0bb6c-d73b-499e-be57-32c25c7eca5b tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.731726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.733308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.734122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.734769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.735601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.738521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.776021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.776941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.777323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.777839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] PATH_INFO: `/v3/users/07d598811ae8427c982bbea05de22daf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.779559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d598811ae8427c982bbea05de22daf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.780374 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2086/4127] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:05:29 2022] POST /identity/v3/projects => generated 332 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:29.783656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:29.786511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:29.854208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d598811ae8427c982bbea05de22daf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:29.855272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d598811ae8427c982bbea05de22daf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:29.868334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d598811ae8427c982bbea05de22daf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:29.885746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d598811ae8427c982bbea05de22daf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:29.887260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d598811ae8427c982bbea05de22daf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:29.930590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50094a88-4457-4e85-a6e5-36337b0c35a3 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:29.933810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2042/4128] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:29 2022] DELETE /identity/v3/users/07d598811ae8427c982bbea05de22daf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:29.938323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:29.939096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:29.939393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:29.939673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:29.940321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:29.943341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.014037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2087/4129] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:29 2022] POST /identity/v3/projects => generated 358 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:30.018270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.019731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:30.033370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:30.150971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030e0ce-63f1-44e0-9e02-3cb6e8cc491c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.274840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.275479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.275764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.276043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.276768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:30.277814 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:30.277814 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:05:30.277814 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:05:30.279486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.326842 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2043/4130] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:05:30 2022] GET /identity/v3/projects => generated 8416 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:30.332116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.346814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d366f-0138-4e99-b06a-d3bfd70e0c7c tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.356799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.357604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.357988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.358389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.364174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:30.364652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:30.367665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.410801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2088/4131] 10.222.0.52 () {58 vars in 1158 bytes} [Sat Sep 24 21:05:30 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:30.411433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.429203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996e6d-8a21-4284-93cc-5e6ae2c01ee2 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.442617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.444018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.444346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.444572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.445324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Adding query filter params (parent_id=0544e62fb1144a5e935a05033d678d68) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:30.445636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:30.450634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.520877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2044/4132] 10.222.0.52 () {58 vars in 1216 bytes} [Sat Sep 24 21:05:30 2022] GET /identity/v3/projects?parent_id=0544e62fb1144a5e935a05033d678d68 => generated 498 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:30.525929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.545190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cb2d0-dcf7-4316-87f6-36b6d7fce176 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.556559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.557556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.559420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.559821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects/c1a9be6408ad48c298f7013509fe9cc4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.561210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c1a9be6408ad48c298f7013509fe9cc4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:30.563342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.636967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a9be6408ad48c298f7013509fe9cc4', 'request_id': 'req-0e6d6536-0f15-4cd5-9daf-d1426b5db822'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:30.639306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:30.648059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a9be6408ad48c298f7013509fe9cc4', 'request_id': 'req-0e6d6536-0f15-4cd5-9daf-d1426b5db822'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:30.658962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a9be6408ad48c298f7013509fe9cc4', 'request_id': 'req-0e6d6536-0f15-4cd5-9daf-d1426b5db822'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:30.699077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] The token cache is being invalidate because project c1a9be6408ad48c298f7013509fe9cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:30.700039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:30.702246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:30.710706 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2089/4133] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:30 2022] DELETE /identity/v3/projects/c1a9be6408ad48c298f7013509fe9cc4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:30.714828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.716612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:30.727016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:30.729418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:30.740724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef50ac4-7263-490f-b146-4876f2ea2863 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.857112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6536-0f15-4cd5-9daf-d1426b5db822 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:30.875080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.875718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.875997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.876270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.879587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:30.913948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:30.916746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:30.918133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:30.918752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects/0544e62fb1144a5e935a05033d678d68` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:30.920259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0544e62fb1144a5e935a05033d678d68)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:30.923367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:30.977264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:30.993775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0544e62fb1144a5e935a05033d678d68', 'request_id': 'req-5ba815d4-e859-4f38-bfaa-58e3beda03e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.009629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.012386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.012758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0544e62fb1144a5e935a05033d678d68', 'request_id': 'req-5ba815d4-e859-4f38-bfaa-58e3beda03e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.021486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0544e62fb1144a5e935a05033d678d68', 'request_id': 'req-5ba815d4-e859-4f38-bfaa-58e3beda03e0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.024350 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2090/4134] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:31.025181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.026560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.038414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] The token cache is being invalidate because project 0544e62fb1144a5e935a05033d678d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:31.038763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.039530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.042629 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2045/4135] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:30 2022] DELETE /identity/v3/projects/0544e62fb1144a5e935a05033d678d68 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:31.045792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.047699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.115839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ce59a-bbba-4c6b-b72d-77c3480ea1ce service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:31.125081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba815d4-e859-4f38-bfaa-58e3beda03e0 tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:31.130029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2c02fedb1343c187c72f0c572b3a3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07a8c1f0c0d6431d858377a0ea0adf39', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:31.131520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.131809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.132084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] PATH_INFO: `/v3/projects/7c38bf230af9440ca331bd57cad8cc79` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.133424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7c38bf230af9440ca331bd57cad8cc79)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.135266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.155923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:31.196998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:31.197465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.197750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.198167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] PATH_INFO: `/v3/projects/7309c3d3d7e542b1a000490468edec55` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.199321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7309c3d3d7e542b1a000490468edec55)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.202431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.210860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c38bf230af9440ca331bd57cad8cc79', 'request_id': 'req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.226032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c38bf230af9440ca331bd57cad8cc79', 'request_id': 'req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.244477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c38bf230af9440ca331bd57cad8cc79', 'request_id': 'req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.266088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] The token cache is being invalidate because project 7c38bf230af9440ca331bd57cad8cc79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:31.266331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.269768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.276826 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2046/4136] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:31 2022] DELETE /identity/v3/projects/7c38bf230af9440ca331bd57cad8cc79 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:31.278450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.281534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.321342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7309c3d3d7e542b1a000490468edec55', 'request_id': 'req-1ae63e5d-9a14-4e80-bda2-f188cb626910'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.329981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7309c3d3d7e542b1a000490468edec55', 'request_id': 'req-1ae63e5d-9a14-4e80-bda2-f188cb626910'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.354999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7309c3d3d7e542b1a000490468edec55', 'request_id': 'req-1ae63e5d-9a14-4e80-bda2-f188cb626910'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.381379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] The token cache is being invalidate because project 7309c3d3d7e542b1a000490468edec55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:31.382222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.382471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1ae63e5d-9a14-4e80-bda2-f188cb626910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.387253 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2091/4137] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:31 2022] DELETE /identity/v3/projects/7309c3d3d7e542b1a000490468edec55 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:31.390109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fca-8f98-4768-8732-4d7c1e8e57b5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.390648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fca-8f98-4768-8732-4d7c1e8e57b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.390957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b0fca-8f98-4768-8732-4d7c1e8e57b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.441122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-402b0fca-8f98-4768-8732-4d7c1e8e57b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:31.444161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-402b0fca-8f98-4768-8732-4d7c1e8e57b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.469492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:31.507673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e293e7-ec64-4c1c-9ffd-5a78cf7883bf tempest-ListProjectsTestJSON-717841935 tempest-ListProjectsTestJSON-717841935-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:31.628389 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2092/4138] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:31.636201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.636555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.636943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.675554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:31.676865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.693389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:31.694341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.694838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.695110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] PATH_INFO: `/v3/users/63f83c27e4fa4658ab097d5de57ab6cb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.696902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f83c27e4fa4658ab097d5de57ab6cb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.702386 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2093/4139] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:31.706479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.710757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.727002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac25b07-518c-4976-b08e-8143048870c8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:31.733778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:31.734397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.734715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.734993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.735650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:31.735959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.737579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.754374 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2094/4140] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:05:31.755331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.766166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec51325-6cb8-4472-85d4-92516eb34e41 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:31.773065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:31.773602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.773886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.774162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.774990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:31.777312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:31.822563 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2095/4141] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:31 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:31.826857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:31.849987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f83c27e4fa4658ab097d5de57ab6cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.851478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f83c27e4fa4658ab097d5de57ab6cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.863453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f83c27e4fa4658ab097d5de57ab6cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.874351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f83c27e4fa4658ab097d5de57ab6cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.875119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c076f28-6da4-446d-a94f-49fd21e57cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f83c27e4fa4658ab097d5de57ab6cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:31.886337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:31.908527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2047/4142] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:31 2022] DELETE /identity/v3/users/63f83c27e4fa4658ab097d5de57ab6cb => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:31.914407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:31.915070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:31.915351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:31.944322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:31.945582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:31.978128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6cab-5eef-436f-a8d5-4de7f96ecf7a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:32.076151 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2048/4143] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:32.127851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:32.129485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:32.173962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:32.174721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.175084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.175869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.177441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:32.180148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:32.200661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:32.220730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2096/4144] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:31 2022] POST /identity/v3/users => generated 286 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:32.236307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:32.239299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:32.266160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0915bd-3b63-4eb9-b134-708c5de888b4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:32.274196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9879-18b4-472b-91c6-3e1696f4752a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:32.280784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:32.281407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.281682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.281806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:32.281936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.282318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.282655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.283425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.284529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:32.291400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:32.306559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:32.309936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:32.319021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2097/4145] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:32.320685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:32.336810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2049/4146] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:32.346025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471843d-365a-41b3-8e2c-675211f12d5d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:32.363801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:32.365839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.366253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.367013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.369501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:32.372612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4424f19-c949-4a71-9218-957404120f61 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:32.390309 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2098/4147] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:32.394630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:32.410104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c2edc-0c10-485c-b6cf-0c147a1cda9c service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:32.423742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:32.425251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.425929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.426857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] PATH_INFO: `/v3/projects/b9bb8e91bd434e1e91c19ba084855d7c/users/6910dfaa89b04d3b9316992d78e9424c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.457213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bb8e91bd434e1e91c19ba084855d7c, user_id=6910dfaa89b04d3b9316992d78e9424c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:32.461985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300782c9-4607-476e-bc14-db9e119c810b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:32.508404 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2050/4148] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:32 2022] PUT /identity/v3/projects/b9bb8e91bd434e1e91c19ba084855d7c/users/6910dfaa89b04d3b9316992d78e9424c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:32.520056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.520342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.520625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.572447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] MFA Rules not processed for user `6910dfaa89b04d3b9316992d78e9424c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:32.573809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:32.751177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2099/4149] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:32.759067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03c91-9138-4169-94bb-d36c11ba60aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:32.759413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03c91-9138-4169-94bb-d36c11ba60aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:32.759696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03c91-9138-4169-94bb-d36c11ba60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:32.787962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c6b03c91-9138-4169-94bb-d36c11ba60aa None None] MFA Rules not processed for user `6910dfaa89b04d3b9316992d78e9424c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:32.789278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c6b03c91-9138-4169-94bb-d36c11ba60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:32.802049 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2051/4150] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:32.808102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:32.809815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:32.917673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:33.006119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aec6f9-a054-4c2a-a582-866665e281d3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:33.014070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:33.015145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:33.015483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:33.015770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:33.037254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:33.039417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:33.074313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2100/4151] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:33.548498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:33.549306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:33.549779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:33.579809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] MFA Rules not processed for user `6910dfaa89b04d3b9316992d78e9424c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:33.582059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:33.593887 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2052/4152] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:33.665819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:33.668136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:33.820084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7fffd-47f7-4b0a-9c64-0278854170b8 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:33.829058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:33.829819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:33.830793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:33.831772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] PATH_INFO: `/v3/projects/e06cf38e47a34bb38d83c91e2670c2aa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:33.835376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06cf38e47a34bb38d83c91e2670c2aa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:33.841120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:33.938792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06cf38e47a34bb38d83c91e2670c2aa', 'request_id': 'req-7c67b2be-63f3-4321-8eb8-1c241fef1aef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:33.951168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06cf38e47a34bb38d83c91e2670c2aa', 'request_id': 'req-7c67b2be-63f3-4321-8eb8-1c241fef1aef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:33.974583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06cf38e47a34bb38d83c91e2670c2aa', 'request_id': 'req-7c67b2be-63f3-4321-8eb8-1c241fef1aef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.004001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] The token cache is being invalidate because project e06cf38e47a34bb38d83c91e2670c2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:34.004345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.005288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.008706 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2101/4153] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:33 2022] DELETE /identity/v3/projects/e06cf38e47a34bb38d83c91e2670c2aa => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:34.012681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:34.029375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2942a-f261-490e-8f7d-00bf23920261 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:34.220769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:34.222850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:34.319329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:34.320542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:34.321021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:34.321644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] PATH_INFO: `/v3/users/6c2c02fedb1343c187c72f0c572b3a3f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:34.332413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2c02fedb1343c187c72f0c572b3a3f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:34.345277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:34.391719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:34.486531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67b2be-63f3-4321-8eb8-1c241fef1aef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:34.509006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2c02fedb1343c187c72f0c572b3a3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.511319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2c02fedb1343c187c72f0c572b3a3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.514779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:34.521859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:34.522284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:34.522669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:34.528004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:34.537083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2c02fedb1343c187c72f0c572b3a3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.549236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:34.555130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:34.569179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2102/4154] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:05:34.590227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2c02fedb1343c187c72f0c572b3a3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.594521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0ff79dae-3e6c-4bad-9e4b-2539079b83da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2c02fedb1343c187c72f0c572b3a3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:34.650285 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2053/4155] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:34 2022] DELETE /identity/v3/users/6c2c02fedb1343c187c72f0c572b3a3f => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:35.556153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:35.557942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:35.587447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b721d-b734-494f-89ce-0ad7fd5dde1a service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:35.751383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:35.753036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:35.753321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:35.753596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] PATH_INFO: `/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:35.757444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a8c1f0c0d6431d858377a0ea0adf39)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:35.762477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:35.902070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a8c1f0c0d6431d858377a0ea0adf39', 'request_id': 'req-828a9570-f989-48f7-a173-1d0a6e5b3b49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:35.911916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a8c1f0c0d6431d858377a0ea0adf39', 'request_id': 'req-828a9570-f989-48f7-a173-1d0a6e5b3b49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:35.918922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a8c1f0c0d6431d858377a0ea0adf39', 'request_id': 'req-828a9570-f989-48f7-a173-1d0a6e5b3b49'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:35.952095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] The token cache is being invalidate because project 07a8c1f0c0d6431d858377a0ea0adf39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:35.952395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:35.953116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-828a9570-f989-48f7-a173-1d0a6e5b3b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:35.958262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2103/4156] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:35 2022] DELETE /identity/v3/projects/07a8c1f0c0d6431d858377a0ea0adf39 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:35.962091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:35.962636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:35.962988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:35.989425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:35.999963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:36.001760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:36.108536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2054/4157] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:36.116845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.117362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.117562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.162841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:36.163994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:36.174517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2104/4158] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:36.178615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.193565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73a076-97ec-4ecc-87da-565d64c60f25 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.199106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.199654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.199933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.200332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.200852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:36.201155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.202880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.228358 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2055/4159] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:36.233147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.247322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8425-8797-47eb-ab04-b2d1360168a0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.253387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.254260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.254729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.263314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.264035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.270005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.322281 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2105/4160] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:36.327011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.339835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de40e-f083-4aac-bc25-ffa0a76bd8aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.426412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.426981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.427283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.427554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.428737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.430713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.453205 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2056/4161] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:36.459284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.473585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5a51-5c2d-49f9-8090-1b0b9b37c64e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.493185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.495200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.496047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.496047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.497905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.501309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.525656 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2106/4162] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:36.527060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.540978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d88d9-40ff-455a-8a1f-1d22c0063455 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.557951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.559012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.559303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.559819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.561450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.566035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28709603-2d82-4a20-93f1-9a0802b9541d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.613776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2057/4163] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:36.618688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.642811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b55021-cfdf-48e5-8ebb-50b4ae66e467 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.650414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.651476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.651887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.652904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] PATH_INFO: `/v3/projects/ec8fe8a19029404db4827fd80f432804/users/e39d9595ffae4bdfa16529990bf2d25a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.673756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8fe8a19029404db4827fd80f432804, user_id=e39d9595ffae4bdfa16529990bf2d25a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.676639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.680356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:36.694527 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2107/4164] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:36 2022] PUT /identity/v3/projects/ec8fe8a19029404db4827fd80f432804/users/e39d9595ffae4bdfa16529990bf2d25a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:36.704849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.705214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.705576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.740353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] MFA Rules not processed for user `e39d9595ffae4bdfa16529990bf2d25a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:36.741731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:36.819337 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2058/4165] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:36.823837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.894288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368c64b-3bdd-465b-9d69-737085dc03c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:36.900169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:36.900846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:36.901594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:36.901594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:36.902226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:36.904235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:36.937362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:36.938585 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2108/4166] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:36.945193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:36.959512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd7be3-0e31-4058-a182-eaa9d1af42b6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.039797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.041357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.041740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.042177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.043017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.045132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.075292 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2059/4167] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:36 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:05:37.076304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.090658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeeb8c-1c46-45df-a080-4781e4ae7285 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.095363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.095967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.096245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.096517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.097152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.098878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.115943 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2109/4168] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:37.117663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.129724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ccc96-0020-403e-96e3-2e3eda13bf0a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.135177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.136125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.137436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.137815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] PATH_INFO: `/v3/projects/9c55e536473c446c88290de31f258cec/users/1631258acb2145a2898c993ff5899947/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.169051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c55e536473c446c88290de31f258cec, user_id=1631258acb2145a2898c993ff5899947, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.174419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.198184 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2060/4169] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:37 2022] PUT /identity/v3/projects/9c55e536473c446c88290de31f258cec/users/1631258acb2145a2898c993ff5899947/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:37.199035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.212007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c34d-493c-4043-956d-8d5188614f40 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.291213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.291901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.292182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.292449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.293089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.295027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dce613-4e08-4615-b929-e8f07b05b7ff admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.309509 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2110/4170] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:37.313326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.327761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa1ca-5ea0-4afa-a17d-3b765cd86dd2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.333145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.334938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.335156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.335521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] PATH_INFO: `/v3/projects/9c55e536473c446c88290de31f258cec/users/1631258acb2145a2898c993ff5899947/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.337889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c55e536473c446c88290de31f258cec, user_id=1631258acb2145a2898c993ff5899947, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.340846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.354118 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2061/4171] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:37 2022] PUT /identity/v3/projects/9c55e536473c446c88290de31f258cec/users/1631258acb2145a2898c993ff5899947/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:37.366408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.366761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.366990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.402079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] MFA Rules not processed for user `1631258acb2145a2898c993ff5899947`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:37.403346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:37.483980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:37.513321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2111/4172] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:37.518245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.535739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a3a06-5860-48a1-a0cf-a5988e94bf7e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.541458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.542368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.542688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.542965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.543629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.544867 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:37.544867 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:05:37.544867 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:05:37.546759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ff8fd-3f0a-4ebf-a691-8215ca2a8d1c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.574914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2062/4173] 10.222.0.52 () {60 vars in 1185 bytes} [Sat Sep 24 21:05:37 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:37.576092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.607121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b13f-f60a-4bfb-9c46-3520949ecc2f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.607410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b13f-f60a-4bfb-9c46-3520949ecc2f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.608568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2fd3a-2f4c-48b0-86b1-51c08298eab3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.610082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b13f-f60a-4bfb-9c46-3520949ecc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.628787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.629462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.629744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.630021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/46f8bb5e54054b6388ed5008b41c65c6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.634609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=46f8bb5e54054b6388ed5008b41c65c6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.639509 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:37.639509 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:05:37.639509 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:05:37.640418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.670870 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2112/4174] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:05:37 2022] GET /identity/v3/OS-OAUTH1/consumers/46f8bb5e54054b6388ed5008b41c65c6 => generated 220 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:37.680896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.735469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16dc14-a013-4e9e-a7db-e7c2c6762887 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.740260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f3a7b13f-f60a-4bfb-9c46-3520949ecc2f None None] MFA Rules not processed for user `762f4e35de9e4083bd0e7a4d0de3c136`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:37.741710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3a7b13f-f60a-4bfb-9c46-3520949ecc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:37.753561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.754224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.754553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.756895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/46f8bb5e54054b6388ed5008b41c65c6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.758853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=46f8bb5e54054b6388ed5008b41c65c6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.760121 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:37.760121 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:05:37.760121 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:05:37.762707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.763428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer 46f8bb5e54054b6388ed5008b41c65c6 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:37.763906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:37.770478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:37.812572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '46f8bb5e54054b6388ed5008b41c65c6', 'request_id': 'req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:37.837727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2113/4175] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:37 2022] DELETE /identity/v3/OS-OAUTH1/consumers/46f8bb5e54054b6388ed5008b41c65c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:37.838994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.840629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:37.853810 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2063/4176] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:37.859501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca65da-ac69-4c7f-ba21-92c56c20e2d8 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.867394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.867394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.867754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.867990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.868568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.868760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.869652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.869971 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:37.869971 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:05:37.869971 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:05:37.871075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.871300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.910943 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2114/4177] 10.222.0.52 () {60 vars in 1185 bytes} [Sat Sep 24 21:05:37 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:37.917677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.924958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] MFA Rules not processed for user `762f4e35de9e4083bd0e7a4d0de3c136`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:37.926621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:37.929707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b9f99-3bf3-4083-a0c8-927273b00e02 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.935319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.937185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.937506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2064/4178] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:37.937988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.938524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.939211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=176e0bddf9c54328a82a4609daa7160c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.940876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.950198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.956566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2115/4179] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:05:37 2022] GET /identity/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c => generated 221 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:37.957489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:37.961413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:37.979581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b151-843b-4eef-8204-aaab676b850f tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:37.985155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:37.985952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:37.986242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:37.986543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:37.987174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=176e0bddf9c54328a82a4609daa7160c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:37.989043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:37.990163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer 176e0bddf9c54328a82a4609daa7160c has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:37.990163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:37.990822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.005207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:38.008565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '176e0bddf9c54328a82a4609daa7160c', 'request_id': 'req-508b3861-265b-4b07-a614-b73675a3161c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.035521 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2116/4180] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:37 2022] DELETE /identity/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:38.038777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.043535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.104456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9a4ba-8b45-4510-a4bb-a416dc8f6b93 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.125081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.126049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.127904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.128359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.132072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.173089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.175184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.215005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2065/4181] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:38.215991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.220412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b3861-265b-4b07-a614-b73675a3161c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.221270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.233482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.234480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.234832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.235209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.240535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.263241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d63f98-8762-4980-a12e-ae911bfa816e service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.275457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.277729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.278015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.278293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.281014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=176e0bddf9c54328a82a4609daa7160c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.282168 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:38.282168 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:05:38.282168 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:05:38.285838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) Sep 24 21:05:38.340531 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 24 21:05:38.341987 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:38.342719 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2066/4182] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:05:38 2022] GET /identity/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c => generated 74 bytes in 126 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:05:38.346204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.360883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.362717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.363794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d3d56-c157-4251-8d72-b2df02ce741c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.381861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.382538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.382826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.383099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.383685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=176e0bddf9c54328a82a4609daa7160c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.385149 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:38.385149 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:05:38.385149 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:05:38.385569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.386174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer 176e0bddf9c54328a82a4609daa7160c has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:38.386933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.388012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.389545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2117/4183] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( Sep 24 21:05:38.428753 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:05:38.429857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2067/4184] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:38 2022] DELETE /identity/v3/OS-OAUTH1/consumers/176e0bddf9c54328a82a4609daa7160c => generated 74 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 21:05:38.434131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.441911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:38.464365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c38ec25-c74e-4dd9-8546-c1b323395ceb service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.482466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.482466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.482466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.482939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.482939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.487904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.517059 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2118/4185] 10.222.0.52 () {60 vars in 1185 bytes} [Sat Sep 24 21:05:38 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:38.521605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.523947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.581901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774453e-38c2-4324-b9d3-de393ec18d83 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.603216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.603844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.604127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.604398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.605053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.606706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.653535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.654878 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2068/4186] 10.222.0.52 () {60 vars in 1185 bytes} [Sat Sep 24 21:05:38 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:38.674797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96016-325e-4422-b13e-8169213a485d tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.684952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.687908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.687908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.687908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.696642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.699107 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:38.699107 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:05:38.699107 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:05:38.702296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.720076 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2119/4187] 10.222.0.52 () {58 vars in 1164 bytes} [Sat Sep 24 21:05:38 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 536 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:38.725040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.738090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241d20f-46c4-4b68-a98d-f9c0265e819c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.745137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.745966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.746514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.746942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/ff780561e88e4617865aeb73c109b7f8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.747645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=ff780561e88e4617865aeb73c109b7f8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.749385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.749886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer ff780561e88e4617865aeb73c109b7f8 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:38.750285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.751243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.766987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'ff780561e88e4617865aeb73c109b7f8', 'request_id': 'req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.795546 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2069/4188] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:38 2022] DELETE /identity/v3/OS-OAUTH1/consumers/ff780561e88e4617865aeb73c109b7f8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:38.800025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.803541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:38.833334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34819-d6de-463e-8766-abafef17f5c0 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.859058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.860993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.861344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.861627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/a6b603cfd06d49f5b43b671651616c03` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.862231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=a6b603cfd06d49f5b43b671651616c03)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.869486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:38.870056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer a6b603cfd06d49f5b43b671651616c03 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:38.870354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.872454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.897089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'a6b603cfd06d49f5b43b671651616c03', 'request_id': 'req-b9127ce1-f091-425b-b6bf-ee2057862129'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:38.932747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2120/4189] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:38 2022] DELETE /identity/v3/OS-OAUTH1/consumers/a6b603cfd06d49f5b43b671651616c03 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:38.938006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:38.957093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ecb4e-fe15-4b78-a0e4-bb0a31d4fc6c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:38.964601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:38.965402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:38.965798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:38.966182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:38.966992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:38.968734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.002004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:39.003914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:39.013481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2070/4190] 10.222.0.52 () {60 vars in 1185 bytes} [Sat Sep 24 21:05:38 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:39.023290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9127ce1-f091-425b-b6bf-ee2057862129 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:39.029354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:39.030342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.030907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.031567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.032485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=2442c66d81b64662b4e29ca8bea49880)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:39.034107 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:05:39.034107 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:05:39.034107 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:05:39.044100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.080390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:39.080575 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2121/4191] 10.222.0.52 () {60 vars in 1285 bytes} [Sat Sep 24 21:05:39 2022] PATCH /identity/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880 => generated 220 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:39.096758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2451b6-f141-44ec-8664-8c9e837d3325 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:39.103800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:39.104561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.105124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.114040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.114040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=2442c66d81b64662b4e29ca8bea49880)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:39.115328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.125927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:39.137488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2071/4192] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:05:39 2022] GET /identity/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880 => generated 220 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:39.140098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5084e-6f45-4d37-8a07-0fe105e9aefa tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:39.148081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1631258acb2145a2898c993ff5899947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c55e536473c446c88290de31f258cec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:39.148643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.148996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.149445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.150107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=2442c66d81b64662b4e29ca8bea49880)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:39.153702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.153702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invalidating token cache because consumer 2442c66d81b64662b4e29ca8bea49880 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:39.153702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.153702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.170830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4 tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': '2442c66d81b64662b4e29ca8bea49880', 'request_id': 'req-fc70f8a9-8cde-4a1f-9458-7704a5d281e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.197128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2122/4193] 10.222.0.52 () {58 vars in 1266 bytes} [Sat Sep 24 21:05:39 2022] DELETE /identity/v3/OS-OAUTH1/consumers/2442c66d81b64662b4e29ca8bea49880 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:39.203101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:39.204905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:39.323538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d2e05-cb38-4680-811f-c18f888b245c tempest-OAUTHConsumersV3Test-2045280059 tempest-OAUTHConsumersV3Test-2045280059-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:39.330558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:39.331212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.331493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.332425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] PATH_INFO: `/v3/users/e39d9595ffae4bdfa16529990bf2d25a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.333617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39d9595ffae4bdfa16529990bf2d25a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:39.336007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.381367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:39.412277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39d9595ffae4bdfa16529990bf2d25a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.413290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39d9595ffae4bdfa16529990bf2d25a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.425762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39d9595ffae4bdfa16529990bf2d25a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.456991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39d9595ffae4bdfa16529990bf2d25a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.457956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39d9595ffae4bdfa16529990bf2d25a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:39.493770 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2072/4194] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:39 2022] DELETE /identity/v3/users/e39d9595ffae4bdfa16529990bf2d25a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:39.496047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.496810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.497307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.535602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:39.537766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:39.721337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2123/4195] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:39.731384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:39.732983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:39.907491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40629bf-a32d-4e26-a7d8-728d41aebbba admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:39.930064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:39.930711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:39.930992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:39.931263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:39.951401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:39.953135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:39.985296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2073/4196] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:41.485658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:41.487341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:41.514416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242451f-b4dd-4196-8af5-fb9f7fa172c2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:41.520139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:41.521054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:41.521535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:41.522341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] PATH_INFO: `/v3/projects/ec8fe8a19029404db4827fd80f432804` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:41.523759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8fe8a19029404db4827fd80f432804)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:41.525880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:41.645783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8fe8a19029404db4827fd80f432804', 'request_id': 'req-0468278c-421b-4566-90ad-473c5a11b313'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:41.654523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8fe8a19029404db4827fd80f432804', 'request_id': 'req-0468278c-421b-4566-90ad-473c5a11b313'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:41.667287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8fe8a19029404db4827fd80f432804', 'request_id': 'req-0468278c-421b-4566-90ad-473c5a11b313'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:41.684210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=11,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:41.720620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] The token cache is being invalidate because project ec8fe8a19029404db4827fd80f432804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:41.720921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:41.723536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:41.727766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2124/4197] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:41 2022] DELETE /identity/v3/projects/ec8fe8a19029404db4827fd80f432804 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:41.731626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:41.754018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d28a1f-c115-4818-9f57-9f1af7538abe service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:41.795567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:41.797082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:41.941718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:41.942631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:41.942888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:41.943168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] PATH_INFO: `/v3/users/1631258acb2145a2898c993ff5899947` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:41.945659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1631258acb2145a2898c993ff5899947)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:41.953041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:42.057517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468278c-421b-4566-90ad-473c5a11b313 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:42.065478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:42.066460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:42.066994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:42.067977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:42.092363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:42.108688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1631258acb2145a2898c993ff5899947'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:42.110918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1631258acb2145a2898c993ff5899947'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:42.125274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:42.144353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1631258acb2145a2898c993ff5899947'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:42.177670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1631258acb2145a2898c993ff5899947'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:42.178905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1631258acb2145a2898c993ff5899947'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:42.229199 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2074/4198] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:41 2022] DELETE /identity/v3/users/1631258acb2145a2898c993ff5899947 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:42.303519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:42.305151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee8bb8-1b1a-4968-8ea2-b1db61d93432 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:42.321764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2125/4199] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:05:43.441245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:43.448937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:43.569062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67cf1e-b9d0-450c-b497-e3cd22c894ea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:43.574747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:43.575668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:43.576124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:43.576563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] PATH_INFO: `/v3/projects/9c55e536473c446c88290de31f258cec` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:43.578436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c55e536473c446c88290de31f258cec)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:43.580816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:43.679750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c55e536473c446c88290de31f258cec', 'request_id': 'req-c4dadefa-4ca2-4012-88a1-0e261e2aa508'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:43.688766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c55e536473c446c88290de31f258cec', 'request_id': 'req-c4dadefa-4ca2-4012-88a1-0e261e2aa508'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:43.707959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c55e536473c446c88290de31f258cec', 'request_id': 'req-c4dadefa-4ca2-4012-88a1-0e261e2aa508'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:43.726708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] The token cache is being invalidate because project 9c55e536473c446c88290de31f258cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:43.727205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:43.728143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c4dadefa-4ca2-4012-88a1-0e261e2aa508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:43.738003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2075/4200] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:43 2022] DELETE /identity/v3/projects/9c55e536473c446c88290de31f258cec => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:43.743971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:43.744791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:43.746065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:43.771310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:43.773124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:43.944776 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2126/4201] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:43.954481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:43.959902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:43.960453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:43.990877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:43.992926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:44.016604 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2076/4202] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:44.016918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.035164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3c64-05fe-4ed7-a1d0-b1b09aad221f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.050591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.051285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.051921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.052482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.053087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:44.053420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.055023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.103748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2127/4203] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:05:44.108829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.132446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3299337-37b3-4066-ad5d-1d30737a1d32 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.144071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.144788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.145139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.145481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.147668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.155011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.229138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2077/4204] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:44 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:44.229854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.244462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe6046-6792-46a8-b568-11eec35d8c47 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.323959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:44.368677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.369414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.369710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.371087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.371724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.373984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.440643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.443875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2128/4205] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:44 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:44.468238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b1521-5d08-4809-894d-84104a867828 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.508016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.508647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.508924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.509194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.526791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.528978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.563413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.564043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2078/4206] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:44.581929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083dea-443e-4610-a059-e6f919209175 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.590902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.591986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.592641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.593304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] PATH_INFO: `/v3/projects/bc9295212d1946f7a750046f46a8d61d/users/acbd3eff760c4363a57246cf370df697/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.618966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9295212d1946f7a750046f46a8d61d, user_id=acbd3eff760c4363a57246cf370df697, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.628687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.657672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2129/4207] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:44 2022] PUT /identity/v3/projects/bc9295212d1946f7a750046f46a8d61d/users/acbd3eff760c4363a57246cf370df697/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:44.663346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.680631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4437-6aba-4e18-bc97-d81a96e7312c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.700309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:44.776898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.777158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.777506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.777917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.778713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.781085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6653f0-81e6-41db-8854-3212edd8f988 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.795055 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2079/4208] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:44.798902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:44.812305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e343b0-6105-40e2-bd53-7e07490c0e91 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:44.828842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:44.830135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.830392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.830833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] PATH_INFO: `/v3/projects/bc9295212d1946f7a750046f46a8d61d/users/acbd3eff760c4363a57246cf370df697/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.833789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9295212d1946f7a750046f46a8d61d, user_id=acbd3eff760c4363a57246cf370df697, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:44.836539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:44.849352 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2130/4209] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:44 2022] PUT /identity/v3/projects/bc9295212d1946f7a750046f46a8d61d/users/acbd3eff760c4363a57246cf370df697/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:44.862669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:44.862982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:44.863393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:44.931765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] MFA Rules not processed for user `acbd3eff760c4363a57246cf370df697`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:44.941313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:45.041244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2080/4210] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:45.042381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.151926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ba8-da0e-43c7-b780-9d3c8a4f195f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.160843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.162751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.163273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.163678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.164447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.166476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.206766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2131/4211] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:45 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:45.211732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.222438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab404b-36f6-4707-9192-7393439fdf1b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.307860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.308589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.308939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.309281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.310020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.312081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.337500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2081/4212] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:45 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:45.341296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.352322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a07ac-6e03-4475-a6fd-f50e97a6dd04 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.357922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.358668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.359019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.359364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.360079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.361810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.378847 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2132/4213] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:45.381338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.400062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bccb6-81cc-4e64-af1f-7e8e92713fba admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.405037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.405581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.405865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.406196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.406878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.409157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825e351-f6fd-4dea-a4ec-09427d9efc79 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.420968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2082/4214] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:45.427094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.438895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b759f-e44e-432e-a3fc-0f3b7fb4e7c2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.444671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.445661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.445661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.446088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] PATH_INFO: `/v3/projects/6736cfe390c34d65be67476dd631786f/users/a96caefd3db34e2f94925779e65c8d9e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.468136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6736cfe390c34d65be67476dd631786f, user_id=a96caefd3db34e2f94925779e65c8d9e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.471402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.495994 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2133/4215] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:45 2022] PUT /identity/v3/projects/6736cfe390c34d65be67476dd631786f/users/a96caefd3db34e2f94925779e65c8d9e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:45.506808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.507366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.507964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.539493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] MFA Rules not processed for user `a96caefd3db34e2f94925779e65c8d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:45.540992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:45.551026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:45.640076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2083/4216] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:45.651888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:45.653715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:45.710037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:45.732407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c938c-f194-46df-a67d-913f1baa7fbd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:45.737109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:45.737993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:45.738375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:45.738845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:45.825637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:45.827262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4da5a-3999-4eb8-b856-f7bdd73151b1 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:45.839788 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2134/4217] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:46.864326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:46.968156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9138f-1712-4530-9df5-4e4c2b6d1a31 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:46.975382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:46.976239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:46.976690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:46.977087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] PATH_INFO: `/v3/users/acbd3eff760c4363a57246cf370df697` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:46.979019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acbd3eff760c4363a57246cf370df697)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:46.982249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:47.062950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acbd3eff760c4363a57246cf370df697'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:47.064106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acbd3eff760c4363a57246cf370df697'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:47.077309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acbd3eff760c4363a57246cf370df697'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:47.093200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acbd3eff760c4363a57246cf370df697'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:47.096044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acbd3eff760c4363a57246cf370df697'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:47.145370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:47.146709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:47.146709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:47.149455 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2084/4218] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:46 2022] DELETE /identity/v3/users/acbd3eff760c4363a57246cf370df697 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:47.186674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:47.188144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:47.352649 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2135/4219] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:47.437257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:47.439527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:47.488289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:47.574186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fda7e-907a-4111-95e8-f1c0fbe10f09 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:47.591214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:47.591837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:47.592116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:47.592383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:47.612324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:47.613880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:47.643614 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2085/4220] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:48.723196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:48.725761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:48.738799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743cdb4-ff93-47c0-8e21-950b53c860e9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:48.746404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:48.747625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:48.747995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:48.749137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] PATH_INFO: `/v3/projects/bc9295212d1946f7a750046f46a8d61d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:48.750700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9295212d1946f7a750046f46a8d61d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:48.752733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:48.835432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9295212d1946f7a750046f46a8d61d', 'request_id': 'req-a1923cdd-c509-4b52-9701-af66a99f3343'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:48.842779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9295212d1946f7a750046f46a8d61d', 'request_id': 'req-a1923cdd-c509-4b52-9701-af66a99f3343'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:48.851524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9295212d1946f7a750046f46a8d61d', 'request_id': 'req-a1923cdd-c509-4b52-9701-af66a99f3343'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:48.870470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] The token cache is being invalidate because project bc9295212d1946f7a750046f46a8d61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:48.870470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:48.870470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:48.871333 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2136/4221] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:48 2022] DELETE /identity/v3/projects/bc9295212d1946f7a750046f46a8d61d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:48.875043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:48.889174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6317b-2a52-4b08-ad1b-57f0e3e65a32 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:48.975248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:48.975990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:48.976358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:48.976712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] PATH_INFO: `/v3/users/a96caefd3db34e2f94925779e65c8d9e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:48.977953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96caefd3db34e2f94925779e65c8d9e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:48.980169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:49.040641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96caefd3db34e2f94925779e65c8d9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:49.041415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96caefd3db34e2f94925779e65c8d9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:49.051441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96caefd3db34e2f94925779e65c8d9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:49.064395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96caefd3db34e2f94925779e65c8d9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:49.067044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3cb27f40-36b1-48c2-b04b-754de9488ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96caefd3db34e2f94925779e65c8d9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:49.089133 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2086/4222] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:48 2022] DELETE /identity/v3/users/a96caefd3db34e2f94925779e65c8d9e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:49.944997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:49.947243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:49.964879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1923cdd-c509-4b52-9701-af66a99f3343 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:49.984744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:50.051298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.055239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.055821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.056300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] PATH_INFO: `/v3/projects/6736cfe390c34d65be67476dd631786f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.062645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6736cfe390c34d65be67476dd631786f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.066046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.184238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6736cfe390c34d65be67476dd631786f', 'request_id': 'req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:50.190639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6736cfe390c34d65be67476dd631786f', 'request_id': 'req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:50.197725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6736cfe390c34d65be67476dd631786f', 'request_id': 'req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:50.214293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] The token cache is being invalidate because project 6736cfe390c34d65be67476dd631786f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:50.214613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:50.215299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3dab1d3e-3ac4-48be-aff8-deb3a3367ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:50.216833 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2137/4223] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:49 2022] DELETE /identity/v3/projects/6736cfe390c34d65be67476dd631786f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:50.221619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.222119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.222236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.242621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:50.243844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:50.309883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:50.335434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2087/4224] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:50.343029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.343318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.343612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.371657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:50.374795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:50.390480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.402688 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2138/4225] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:50.414406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cea27-1dc9-4dc3-9f46-3b1d01c1b12f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.425738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.425738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.425738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.425738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.426330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:50.426330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.429494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.449095 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2088/4226] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:50.450676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.471171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907758-b38b-4713-8c9f-7919c9816f8c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.479584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.481184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.481184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.481439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.481979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.484191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.522356 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2139/4227] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:50 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:50.526606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.538300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24d40e-0e0c-4eb9-b99d-2e699a7afd02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.651582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.655294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.655710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.659381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.660294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.662938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.706338 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2089/4228] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:50 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:50.710854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.728893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cce8d-2b8a-4ada-935d-218b2e5cfbbb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.736502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.738041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.738524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.741377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.741377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.744719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.768142 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2140/4229] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:50.772992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.786012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9632d5f-68fa-4b1b-af20-1e06c69dd248 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.796354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.797115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.798227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.799644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.800589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.803264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9eba79-240e-4441-9a74-e5b6acbc3899 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.824535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2090/4230] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:50.825532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:50.837860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094467-2043-4470-bdab-b16534cd6bfe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:50.847026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:50.847993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.848158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.848670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] PATH_INFO: `/v3/projects/6a4d20c424fa4a7c8fe10ba3e05ada24/users/3b9d49d3cd9f4c298f6fa7453a6c383c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.878881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4d20c424fa4a7c8fe10ba3e05ada24, user_id=3b9d49d3cd9f4c298f6fa7453a6c383c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:50.882612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:50.900966 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2141/4231] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:50 2022] PUT /identity/v3/projects/6a4d20c424fa4a7c8fe10ba3e05ada24/users/3b9d49d3cd9f4c298f6fa7453a6c383c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:05:50.914122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802500e-827c-4cd9-85eb-256b6da1a666 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:50.914545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802500e-827c-4cd9-85eb-256b6da1a666 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:50.914924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802500e-827c-4cd9-85eb-256b6da1a666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:50.976546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6802500e-827c-4cd9-85eb-256b6da1a666 None None] MFA Rules not processed for user `3b9d49d3cd9f4c298f6fa7453a6c383c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:50.979583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6802500e-827c-4cd9-85eb-256b6da1a666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:51.072290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2091/4232] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:51.082103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:51.084101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:51.162766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:51.192276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f258da3-84ac-4df2-842e-016fb39a358c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:51.196440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:51.197203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:51.198116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:51.198400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:51.214424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:51.217161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:51.238408 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2142/4233] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:51.449365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099693a-3b6f-41c9-810a-066c9b555089 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:51.450239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099693a-3b6f-41c9-810a-066c9b555089 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:51.450775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099693a-3b6f-41c9-810a-066c9b555089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:51.497387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a099693a-3b6f-41c9-810a-066c9b555089 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:51.500984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a099693a-3b6f-41c9-810a-066c9b555089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:51.578785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:51.637535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2092/4234] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:51.706919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:51.710046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:51.871744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b34496-9b60-46c1-8272-0fa4296b2cbd service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:51.888595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:51.889220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:51.889786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:51.890264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:51.913804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:51.916928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:51.939349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2143/4235] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:05:52.516458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ec6b-ac29-4c5f-b625-386c65bfebe1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:52.517018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ec6b-ac29-4c5f-b625-386c65bfebe1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:52.517018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5ec6b-ac29-4c5f-b625-386c65bfebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:52.540196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1df5ec6b-ac29-4c5f-b625-386c65bfebe1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:52.541412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1df5ec6b-ac29-4c5f-b625-386c65bfebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:52.549385 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2093/4236] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:52.561654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:52.584187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc24b44-eadd-4375-a379-7884bbdfcafc service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:52.592259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:52.593515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:52.594117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:52.594639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:52.607099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:52.608829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:52.620180 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2144/4237] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:05:53.220574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d28469-544a-45d2-b541-7e8fb2c8789e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:53.220862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d28469-544a-45d2-b541-7e8fb2c8789e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:53.221141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d28469-544a-45d2-b541-7e8fb2c8789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:53.245267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-10d28469-544a-45d2-b541-7e8fb2c8789e None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:53.246207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-10d28469-544a-45d2-b541-7e8fb2c8789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:53.255094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2094/4238] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:53.282139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:53.303868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dce37-b1d6-4586-a7db-5b236c93f2b0 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:53.310687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:53.311899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:53.312360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:53.312806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:53.326578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:53.329260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:53.342865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2145/4239] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:05:53.613649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:53.614763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:53.615283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:53.643633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:53.645562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:53.666937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2095/4240] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:53.675291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:53.694367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e0629-3214-4d6e-8357-22b51fa32d00 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:53.700384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:53.701320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:53.701831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:53.703108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:53.714100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:53.716189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435c42a-9a35-46a3-9312-5c21d38db9df service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:53.727016 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2146/4241] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:05:54.160150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:54.167097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:54.378742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4e0f1-ff3d-4781-83b6-4dcfc8153b81 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:54.404024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:54.404707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:54.423527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:54.424106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:54.430283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:54.493580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:54.495482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:54.524580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2096/4242] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:54.555528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c7420-677d-4621-91d2-0df866c15880 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:54.555810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c7420-677d-4621-91d2-0df866c15880 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:54.556088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c7420-677d-4621-91d2-0df866c15880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:54.634449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-879c7420-677d-4621-91d2-0df866c15880 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:54.636288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-879c7420-677d-4621-91d2-0df866c15880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:54.657014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2147/4243] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:54.672472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:54.687508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfb54c-31db-4923-b051-0e73221dce79 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:54.694583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:54.695502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:54.695782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:54.696334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:54.705768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:54.709290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:54.720914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2097/4244] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:05:54.933568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:54.934074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:54.934392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:54.956357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:54.957746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:54.965612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2148/4245] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:54.978768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:54.992208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9346-45ec-4285-bfd5-7a064b2f2ce9 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:54.997313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:54.997970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:54.998316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:54.998681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:55.007507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:55.009165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57238c-2408-4f6d-bed3-19e07b944a9f service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:55.018038 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2098/4246] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:05:55.156623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:55.234975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158dc86d-7a8f-4360-9a3e-8d352492b5d1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:55.243808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:55.244612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:55.245173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:55.245755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] PATH_INFO: `/v3/users/3b9d49d3cd9f4c298f6fa7453a6c383c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:55.247184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9d49d3cd9f4c298f6fa7453a6c383c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:55.249306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:55.282001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:55.311232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9d49d3cd9f4c298f6fa7453a6c383c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:55.312306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9d49d3cd9f4c298f6fa7453a6c383c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:55.322242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9d49d3cd9f4c298f6fa7453a6c383c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:55.340694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9d49d3cd9f4c298f6fa7453a6c383c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:55.341944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9d49d3cd9f4c298f6fa7453a6c383c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:55.365632 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2149/4247] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:55 2022] DELETE /identity/v3/users/3b9d49d3cd9f4c298f6fa7453a6c383c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:55.370863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:55.371403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:55.371676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:55.411708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:55.413323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:55.519759 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2099/4248] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:55.538409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:55.547442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:55.687892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72142dd5-a84a-4f16-be49-abd7e271b9f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:55.694176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:55.695017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:55.695740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:55.696028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:55.714076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:55.715958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbef19-76e3-4394-b7c1-06f4335f9a1c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:55.729579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2150/4249] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:56.143812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.145747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:56.159809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b618d4-fd16-466f-b2c7-dd1ad44d8165 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.167127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.167926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.168461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.168997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] PATH_INFO: `/v3/projects/6a4d20c424fa4a7c8fe10ba3e05ada24` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.170674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4d20c424fa4a7c8fe10ba3e05ada24)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.175597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.195107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:56.230089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4d20c424fa4a7c8fe10ba3e05ada24', 'request_id': 'req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:56.236306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4d20c424fa4a7c8fe10ba3e05ada24', 'request_id': 'req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:56.245046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4d20c424fa4a7c8fe10ba3e05ada24', 'request_id': 'req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:56.257826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] The token cache is being invalidate because project 6a4d20c424fa4a7c8fe10ba3e05ada24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:56.258104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:56.258790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602c7fb6-56d8-4c4c-a03d-b48c04baecbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:56.260283 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2100/4250] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:56 2022] DELETE /identity/v3/projects/6a4d20c424fa4a7c8fe10ba3e05ada24 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:56.265900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.266459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.266459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.288670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:56.289903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:56.359331 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2151/4251] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:56.360975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.361291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.361571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.382388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:56.383666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:56.393268 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2101/4252] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:56.398132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.408184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15319da-45b3-420a-a856-1a73e342c27b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.413275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.413955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.414300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.416001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.416001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:56.416001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.417548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.431919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2152/4253] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:56.440678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.464048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f510276-b4cd-4f6c-97e7-39f28c42f9e6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.490377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.491118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.491401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.491674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.492316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.494243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.524968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2102/4254] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:56 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:56.529186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.540030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad473df2-60d3-416e-a7b9-680bfbe82af4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.574595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:56.627553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.628169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.628447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.628740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.630647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.633065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.659933 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2153/4255] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:56 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:56.665642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.685468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec58030-fe9d-4567-9de1-a37ac598181a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.693291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.694258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.695005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.696058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.697138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.699535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.719203 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2103/4256] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:56.722927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.741944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da39e6d-2614-4112-8df8-4445c858015e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.752040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.753510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.756769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.757782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.759142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.761834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385fe0-4243-40ed-9909-f9f0f2d2d993 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.774793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2154/4257] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:56.779316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.792399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb6236-f23b-43c8-9add-de8a4c685499 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:56.799957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:56.800633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.800905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.801427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] PATH_INFO: `/v3/projects/90f0ec3457ce42e8a58c9113e789c566/users/37e11885fcb943c0a686a62ec8bebafa/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.821068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f0ec3457ce42e8a58c9113e789c566, user_id=37e11885fcb943c0a686a62ec8bebafa, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:56.824118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:56.851080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2104/4258] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:56 2022] PUT /identity/v3/projects/90f0ec3457ce42e8a58c9113e789c566/users/37e11885fcb943c0a686a62ec8bebafa/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:56.862062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:56.862450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:56.862864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:56.903825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] MFA Rules not processed for user `37e11885fcb943c0a686a62ec8bebafa`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:56.905070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:56.972316 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2155/4259] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:56.985676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:56.987394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:57.048699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008ea89-7798-4d24-9c3a-d8da92a7f3b5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:57.053077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:57.053860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:57.054148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:57.054422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:57.063363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:57.064918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebb728-5202-4338-89fa-7174f2c6091d service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:57.075009 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2105/4260] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:57.178931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:57.244643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721359c8-191f-46c1-8354-a9e3aaac5d58 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:57.256193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:57.256887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:57.257307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:57.257587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] PATH_INFO: `/v3/users/37e11885fcb943c0a686a62ec8bebafa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:57.259114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e11885fcb943c0a686a62ec8bebafa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:57.264764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:57.338379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:57.346215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e11885fcb943c0a686a62ec8bebafa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:57.347122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e11885fcb943c0a686a62ec8bebafa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:57.361694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e11885fcb943c0a686a62ec8bebafa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:57.383262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e11885fcb943c0a686a62ec8bebafa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:57.384275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e11885fcb943c0a686a62ec8bebafa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:57.402477 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2156/4261] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:05:57 2022] DELETE /identity/v3/users/37e11885fcb943c0a686a62ec8bebafa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:57.404149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:57.404936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:57.405213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:57.434039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:57.435488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:57.495152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:57.530894 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2106/4262] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:57.541367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:57.543990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:57.633493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593951d3-3506-4330-aa6b-92dc8dd9ec29 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:57.651567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:57.652256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:57.652541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:57.652870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:57.671836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:57.673463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0113c68-9426-4cbd-a93a-8cc1e19c24c6 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:57.684886 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2157/4263] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:05:58.437450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:58.440838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:58.462000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4a494-2a23-46e7-9bb1-99f14b00da8c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:58.471571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:58.474985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:58.475959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:58.476454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] PATH_INFO: `/v3/projects/90f0ec3457ce42e8a58c9113e789c566` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:58.480083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f0ec3457ce42e8a58c9113e789c566)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:58.483285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:58.572761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f0ec3457ce42e8a58c9113e789c566', 'request_id': 'req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:58.579084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f0ec3457ce42e8a58c9113e789c566', 'request_id': 'req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:58.588405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f0ec3457ce42e8a58c9113e789c566', 'request_id': 'req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:58.602736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] The token cache is being invalidate because project 90f0ec3457ce42e8a58c9113e789c566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:05:58.603020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:58.603675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f38c540-0ce0-4750-aca1-f3f5b173bf0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:05:58.605197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2107/4264] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:05:58 2022] DELETE /identity/v3/projects/90f0ec3457ce42e8a58c9113e789c566 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:58.610404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:58.610930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:58.611063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:58.634766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:58.636148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:58.748927 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2158/4265] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:05:58.758382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:58.758814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:58.759191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:58.789954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:58.791925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:58.804270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2108/4266] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:58.808167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:58.841706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a499-e590-4e9f-8f73-73a550479b28 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:58.867127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:58.867127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:58.867520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:58.867520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:58.868091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:05:58.872018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:58.872618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:58.912792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2159/4267] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:05:58.916937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:58.927488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183027fb-ff15-47f6-82da-3870e08c0ba7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:58.935361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:58.937459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:58.938074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:58.938607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:58.939504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:58.941679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:58.983857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2109/4268] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:58 2022] POST /identity/v3/projects => generated 399 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:58.985285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:58.999105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c01a5-b476-40fa-b2c3-bd5399b10698 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.068614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:59.096857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.097652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.097794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.098613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.099233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.104487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.156528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.159724 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2160/4269] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:58 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:59.171409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e78036-f796-4e50-8aa4-e529922cae87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.176753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.177643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.178073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.178641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.179438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.181223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.199831 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2110/4270] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:59.203443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.219249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da71b-a74a-4503-ac02-bad206f002ca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.229405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.230029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.230308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.230603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.231245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.232850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66a09b-1168-42c2-9944-e634f501575a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.266686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2161/4271] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:05:59.280327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.296002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166581fd-56fc-49e6-b7e8-5fb5713d990b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.305519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.306440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.306742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.318189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] PATH_INFO: `/v3/projects/969b9631431c4f5ebade8c3e2a55aed4/users/0244849b631a43278742a5fd6bca5141/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.337510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=969b9631431c4f5ebade8c3e2a55aed4, user_id=0244849b631a43278742a5fd6bca5141, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.348738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.370455 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2111/4272] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:59 2022] PUT /identity/v3/projects/969b9631431c4f5ebade8c3e2a55aed4/users/0244849b631a43278742a5fd6bca5141/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:59.381874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.382156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.382426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.411678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] MFA Rules not processed for user `0244849b631a43278742a5fd6bca5141`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:05:59.413074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:05:59.480066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2162/4273] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:05:59.484467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.549585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240a60d-6b7d-465f-88e8-33c54c8945e5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.555033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.555765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.556138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.556507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.557228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.559256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.566684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:05:59.597073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2112/4274] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:05:59 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:59.598139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.609873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4982dcb-5bf7-4d27-929e-2f62724ed7d4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.710265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.711015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.711372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.711722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.712505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.714555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.772255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2163/4275] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:05:59 2022] POST /identity/v3/users => generated 306 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:05:59.776277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.805966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5da3f-3c25-4e6a-b98d-4938c035d13d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.829654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.834782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.835087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.835368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.836046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.837687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.853160 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2113/4276] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:05:59.854305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.865423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194fd8e-c69f-4576-a49b-91996595cf58 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.870115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:05:59.870800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:05:59.871076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:05:59.871347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] PATH_INFO: `/v3/projects/f77d3c331b7940fabb8680aba4569c02/users/c153f4354803475493e776a6473f90a6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:05:59.891363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77d3c331b7940fabb8680aba4569c02, user_id=c153f4354803475493e776a6473f90a6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:05:59.894694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:05:59.913484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2164/4277] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:05:59 2022] PUT /identity/v3/projects/f77d3c331b7940fabb8680aba4569c02/users/c153f4354803475493e776a6473f90a6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:05:59.917085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:05:59.928369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef91542-1eb7-474d-8fbf-3aa5ec895a4f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:05:59.999353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:00.000028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:00.000312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:00.000580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:00.001216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:00.002861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e56889-48d5-441a-9842-bebb1c11324c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:00.013346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2114/4278] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:00.017539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:00.028988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5a73-cdf7-48c6-baea-e8ebf1cb3a75 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:00.034035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:00.034733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:00.035012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:00.035284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] PATH_INFO: `/v3/projects/f77d3c331b7940fabb8680aba4569c02/users/c153f4354803475493e776a6473f90a6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:00.037258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77d3c331b7940fabb8680aba4569c02, user_id=c153f4354803475493e776a6473f90a6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:00.039947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2745a-9ede-4d68-8d8f-25f6fb203683 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:00.050698 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2165/4279] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:00 2022] PUT /identity/v3/projects/f77d3c331b7940fabb8680aba4569c02/users/c153f4354803475493e776a6473f90a6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:00.061502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:00.061792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:00.062068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:00.109447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] MFA Rules not processed for user `c153f4354803475493e776a6473f90a6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:00.109447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:00.117381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771026b4-00f0-446c-8370-d7379f474564 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:00.117663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771026b4-00f0-446c-8370-d7379f474564 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:00.117935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771026b4-00f0-446c-8370-d7379f474564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:00.139581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-771026b4-00f0-446c-8370-d7379f474564 None None] MFA Rules not processed for user `e557d91b49c143499d809a55e85932fd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:00.141760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-771026b4-00f0-446c-8370-d7379f474564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:00.155987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:00.234565 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2115/4280] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:00.245370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:00.247583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:00.250627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2166/4281] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:00.379392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09500ba-40bb-4361-9415-2f9dd68f18f7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:00.384835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:00.385714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:00.386215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:00.386871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:00.401642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:00.503186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:00.505386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edc682-fbfc-47dd-acd4-f98e5ca9de6b service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:00.517887 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2116/4282] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:01.180692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771026b4-00f0-446c-8370-d7379f474564 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:01.182130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-771026b4-00f0-446c-8370-d7379f474564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:01.261240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771026b4-00f0-446c-8370-d7379f474564 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:01.267541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:01.268281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:01.268567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:01.268846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] PATH_INFO: `/v3/users/34086609542f4d44bcb50c88141d1220` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:01.270178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34086609542f4d44bcb50c88141d1220)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:01.273167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:01.337806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34086609542f4d44bcb50c88141d1220'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:01.338799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34086609542f4d44bcb50c88141d1220'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:01.350459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34086609542f4d44bcb50c88141d1220'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:01.373827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34086609542f4d44bcb50c88141d1220'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:01.374594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34086609542f4d44bcb50c88141d1220'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:01.396116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2167/4283] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:01 2022] DELETE /identity/v3/users/34086609542f4d44bcb50c88141d1220 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:01.401927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:01.402810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:01.403421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:01.427403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:01.429035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:01.509369 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2117/4284] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:01.521466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:01.522969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:01.572991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:01.606388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182307-5ef2-4d9b-8cf5-d5ce81185d58 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:01.611467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:01.612214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:01.612561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:01.612898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:01.623939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:01.625523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:01.662977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2168/4285] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:01.770687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:01.776075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:01.932699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:01.934260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:01.960714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a441796-5ac4-4c25-8df4-2acfe4ab63a2 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:01.972401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:01.973105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:01.974628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:01.974914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] PATH_INFO: `/v3/projects/306f579ca5df4c3fa4d42221439425a7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:01.978349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=306f579ca5df4c3fa4d42221439425a7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:01.980852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:01.993562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce530a82-cfec-4909-b071-b5dd1234f3d9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:02.001562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:02.002648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:02.003085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:02.003530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:02.009433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:02.124400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '306f579ca5df4c3fa4d42221439425a7', 'request_id': 'req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.129574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:02.133300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '306f579ca5df4c3fa4d42221439425a7', 'request_id': 'req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.141378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '306f579ca5df4c3fa4d42221439425a7', 'request_id': 'req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.148784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:02.150560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:02.161380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] The token cache is being invalidate because project 306f579ca5df4c3fa4d42221439425a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:02.162321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.162559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2118/4286] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:02.163358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.166741 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2169/4287] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:01 2022] DELETE /identity/v3/projects/306f579ca5df4c3fa4d42221439425a7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:02.170219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:02.188370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f006893-8849-4ee7-9ff6-35cd7e5dbf2e service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:02.312154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:02.313032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:02.313371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:02.313950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] PATH_INFO: `/v3/users/1aae9d32275345d7ba6e857fc74a2686` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:02.318684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aae9d32275345d7ba6e857fc74a2686)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:02.321628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:02.413777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aae9d32275345d7ba6e857fc74a2686'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.414766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aae9d32275345d7ba6e857fc74a2686'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.426848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aae9d32275345d7ba6e857fc74a2686'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.448251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aae9d32275345d7ba6e857fc74a2686'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.449701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aae9d32275345d7ba6e857fc74a2686'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.496833 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2119/4288] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:02 2022] DELETE /identity/v3/users/1aae9d32275345d7ba6e857fc74a2686 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:02.593167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:02.595013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:02.706584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:02.710445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:02.726907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02d287-8d3c-4e9b-aebb-edf4ecf08cb9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:02.818168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a381e8d-44d9-47b6-8763-4a20e43fe600 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:02.823632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:02.824288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:02.824565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:02.824834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] PATH_INFO: `/v3/users/f05599f233ca45c39ccd351c0432f1ba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:02.826043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05599f233ca45c39ccd351c0432f1ba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:02.828929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:02.836419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:02.881056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:02.895456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05599f233ca45c39ccd351c0432f1ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.895895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:02.896584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05599f233ca45c39ccd351c0432f1ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.896725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:02.896901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:02.897184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:02.900274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:02.906908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05599f233ca45c39ccd351c0432f1ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.915646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05599f233ca45c39ccd351c0432f1ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.916393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05599f233ca45c39ccd351c0432f1ba'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:02.932859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:02.937880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:02.961219 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2170/4289] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:02.967965 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2120/4290] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:02 2022] DELETE /identity/v3/users/f05599f233ca45c39ccd351c0432f1ba => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:03.388394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:03.390368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:03.408110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f43d73-096c-4d74-924e-117712a9e611 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:03.459129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:03.461823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:03.500315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:03.501011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:03.502963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:03.503279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] PATH_INFO: `/v3/projects/2cf31b4dea5d4b19b3e2d01f930427e6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:03.507052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf31b4dea5d4b19b3e2d01f930427e6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:03.511316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:03.608806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf31b4dea5d4b19b3e2d01f930427e6', 'request_id': 'req-498d3280-573e-490c-b330-1d66680dca92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.623877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf31b4dea5d4b19b3e2d01f930427e6', 'request_id': 'req-498d3280-573e-490c-b330-1d66680dca92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.638627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf31b4dea5d4b19b3e2d01f930427e6', 'request_id': 'req-498d3280-573e-490c-b330-1d66680dca92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.645869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e6cf6-2237-43c4-8ae0-af5fdc8955e7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:03.658407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] The token cache is being invalidate because project 2cf31b4dea5d4b19b3e2d01f930427e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:03.658678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.659283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.665006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2171/4291] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:03 2022] DELETE /identity/v3/projects/2cf31b4dea5d4b19b3e2d01f930427e6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:03.666038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:03.667578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:03.741199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:03.741899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:03.742478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:03.742870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] PATH_INFO: `/v3/projects/3ee2eb7dffa647fb89edd0cf7d1cb454` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:03.744113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee2eb7dffa647fb89edd0cf7d1cb454)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:03.746687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:03.775720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d3280-573e-490c-b330-1d66680dca92 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:03.781743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:03.782447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:03.783203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:03.783628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] PATH_INFO: `/v3/users/7d060d99930f4922b358da724a9303cb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:03.785981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d060d99930f4922b358da724a9303cb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:03.788613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:03.808174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:03.821032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:03.831046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee2eb7dffa647fb89edd0cf7d1cb454', 'request_id': 'req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.838261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee2eb7dffa647fb89edd0cf7d1cb454', 'request_id': 'req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.848382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee2eb7dffa647fb89edd0cf7d1cb454', 'request_id': 'req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.853308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d060d99930f4922b358da724a9303cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.854089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d060d99930f4922b358da724a9303cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.865414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d060d99930f4922b358da724a9303cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.871936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] The token cache is being invalidate because project 3ee2eb7dffa647fb89edd0cf7d1cb454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:03.872254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.875722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.887938 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2121/4292] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:03 2022] DELETE /identity/v3/projects/3ee2eb7dffa647fb89edd0cf7d1cb454 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:03.891813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:03.896599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:03.898413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d060d99930f4922b358da724a9303cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.899425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d060d99930f4922b358da724a9303cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:03.923961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2172/4293] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:03 2022] DELETE /identity/v3/users/7d060d99930f4922b358da724a9303cb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:03.981728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf84db6-8f0c-4687-9eea-b04c6b09de2a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.050637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.052039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.052397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.052735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] PATH_INFO: `/v3/users/09443ab1a6ac4b5db4567fa891abe1d1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.054772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09443ab1a6ac4b5db4567fa891abe1d1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.057181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.159779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09443ab1a6ac4b5db4567fa891abe1d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.160625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09443ab1a6ac4b5db4567fa891abe1d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.171702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09443ab1a6ac4b5db4567fa891abe1d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.184600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09443ab1a6ac4b5db4567fa891abe1d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.185680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09443ab1a6ac4b5db4567fa891abe1d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.205307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:04.234086 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2122/4294] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:03 2022] DELETE /identity/v3/users/09443ab1a6ac4b5db4567fa891abe1d1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:04.509704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.511191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.529488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27997-5e36-4931-a5f4-2968c3f4650b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.609706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.610742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.610897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.611158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] PATH_INFO: `/v3/projects/d9836534a7794c98b32eaec0170e83b8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.612360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9836534a7794c98b32eaec0170e83b8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.614333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.615691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.617138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.633780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382c69-383c-4f93-8d3d-37be888c3fc0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.639838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.640598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.641128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.641494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] PATH_INFO: `/v3/projects/d56ba00623954a158eda25112ea04d6d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.642995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56ba00623954a158eda25112ea04d6d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.646110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.698728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9836534a7794c98b32eaec0170e83b8', 'request_id': 'req-cb17ac1f-467b-4522-99a7-5ce162a32960'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.706309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9836534a7794c98b32eaec0170e83b8', 'request_id': 'req-cb17ac1f-467b-4522-99a7-5ce162a32960'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.711574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56ba00623954a158eda25112ea04d6d', 'request_id': 'req-bbb510b3-0279-4491-b14a-8b12691416bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.714598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9836534a7794c98b32eaec0170e83b8', 'request_id': 'req-cb17ac1f-467b-4522-99a7-5ce162a32960'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.718953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56ba00623954a158eda25112ea04d6d', 'request_id': 'req-bbb510b3-0279-4491-b14a-8b12691416bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.728354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56ba00623954a158eda25112ea04d6d', 'request_id': 'req-bbb510b3-0279-4491-b14a-8b12691416bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.730161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] The token cache is being invalidate because project d9836534a7794c98b32eaec0170e83b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:04.730549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.731300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb17ac1f-467b-4522-99a7-5ce162a32960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.733127 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2173/4295] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:04 2022] DELETE /identity/v3/projects/d9836534a7794c98b32eaec0170e83b8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:04.737701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c37d-3594-4b54-961c-09365859aee6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.738088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c37d-3594-4b54-961c-09365859aee6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.739024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c37d-3594-4b54-961c-09365859aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.743464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] The token cache is being invalidate because project d56ba00623954a158eda25112ea04d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:04.743974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.744977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bbb510b3-0279-4491-b14a-8b12691416bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:04.746753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2123/4296] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:04 2022] DELETE /identity/v3/projects/d56ba00623954a158eda25112ea04d6d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:04.751663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c64f8-07c3-48b3-bbf0-d64748f313ec None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.752317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c64f8-07c3-48b3-bbf0-d64748f313ec None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.752647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c64f8-07c3-48b3-bbf0-d64748f313ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.763320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1aa5c37d-3594-4b54-961c-09365859aee6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:04.764761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1aa5c37d-3594-4b54-961c-09365859aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.774557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e84c64f8-07c3-48b3-bbf0-d64748f313ec None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:04.776521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e84c64f8-07c3-48b3-bbf0-d64748f313ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.798972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:04.845214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2124/4297] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:04.845767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2174/4298] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:04.854365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.854682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.854963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.859365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc666-efac-45b3-a021-484259307ea5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.859660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc666-efac-45b3-a021-484259307ea5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.859936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bc666-efac-45b3-a021-484259307ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.876335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:04.877928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.883594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-627bc666-efac-45b3-a021-484259307ea5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:04.884807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-627bc666-efac-45b3-a021-484259307ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:04.889582 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2125/4299] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:04.893880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.894323 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2175/4300] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:04.899380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bc666-efac-45b3-a021-484259307ea5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.908456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a3998-59bc-4a83-b6be-37e8ebd8a0fe None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.909805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bc666-efac-45b3-a021-484259307ea5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.914460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.915097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.915669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.915806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.915966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.916240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.916553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.916918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:04.917157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.917281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.918871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.919208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:04.919737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.922257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.937240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2176/4301] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:06:04.938251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.946034 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2126/4302] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:06:04.947553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:04.962897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55412a-96fd-42a5-8fb2-e05b8bb867a7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.970131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.971434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.971863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.972202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.973470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.975708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:04.976005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823dc7b-c26c-4322-b00f-c5ec41bd2132 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:04.982864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:04.984130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:04.984576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:04.985011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:04.985805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:04.988095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.028869 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2177/4303] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:05.034619 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2127/4304] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:04 2022] POST /identity/v3/projects => generated 407 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:05.034795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.036279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.121690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c9853-c3eb-4ddd-b2c9-c08a652be38b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.126415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:05.280846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed77289-1e9e-40ac-81ab-14114ec82de5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.294594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.303487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.304571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.306109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.306532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.307381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.309935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.310081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.310350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.311029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.312987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.314770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.383595 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2128/4305] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:05 2022] POST /identity/v3/users => generated 304 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:06:05.384079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.403465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2178/4306] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:05 2022] POST /identity/v3/users => generated 293 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:05.407580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.416723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168d54f-47ec-469c-b867-b631e3395964 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.422270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.423137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.423429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.423707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.424359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.426682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.427718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65459-7f8b-4b3c-b5f5-e38da2fbe2fb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.437282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.438116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.438614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.439064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.440113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.441818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.445468 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2129/4307] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:05.449823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.456984 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2179/4308] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:05.461734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.472259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46c62c-bada-4330-9408-a882c202caa4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.481980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2d8f-5629-492a-81b5-43f2a0ac06b2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.483212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.484162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.484403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.484843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.485498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.487415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.489239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.490845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.491222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.491585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.492363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.494697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.502378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2130/4309] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:05.506860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.514083 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2180/4310] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:05.515789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.522147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739198e-1747-4e36-9cc6-35352665971d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.525914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:05.531091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.531417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11618268-d6de-4ce2-aa49-1d3ba975e7be admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.532227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.532510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.533056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] PATH_INFO: `/v3/projects/2ae36fb9a68c48fc9a7888a71146725a/users/66ffbeeec3274f47a826fcc5e541feaa/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.540077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.541014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.541386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.541752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] PATH_INFO: `/v3/projects/ede438e67af3496cae36fc89abc0aba3/users/370b057b12c148cdbb673ba6c06bd6b4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.563381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede438e67af3496cae36fc89abc0aba3, user_id=370b057b12c148cdbb673ba6c06bd6b4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.574193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a3a45-201b-45b0-88e7-877fb85084a7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.575093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae36fb9a68c48fc9a7888a71146725a, user_id=66ffbeeec3274f47a826fcc5e541feaa, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.578082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5a931-fa88-4cd4-b9f6-51470d704ca2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.617554 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2181/4311] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:05 2022] PUT /identity/v3/projects/ede438e67af3496cae36fc89abc0aba3/users/370b057b12c148cdbb673ba6c06bd6b4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:05.621600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2131/4312] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:05 2022] PUT /identity/v3/projects/2ae36fb9a68c48fc9a7888a71146725a/users/66ffbeeec3274f47a826fcc5e541feaa/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:05.622755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.623048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.623324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.633557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.633980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.634874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.666741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:05.668793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:05.694717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] MFA Rules not processed for user `66ffbeeec3274f47a826fcc5e541feaa`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:05.696332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:05.758397 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2182/4313] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:05.820500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.822198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:05.956859 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2132/4314] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:05.958588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:05.967084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49506d-4dad-4fe2-83f6-d67392e38789 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.971207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.971812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.972289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.972558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.976791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2e066-b3dd-4abe-b659-121bf05e889c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:05.987429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:05.988678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:05.989799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:05.990247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:05.991225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.991539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:05.993630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:05.996575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.023305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2183/4315] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:06.037243 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2133/4316] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:05 2022] POST /identity/v3/projects => generated 405 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:06.040955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.051682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539cedd-8d87-4657-9452-373d6787aa99 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.132471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.133090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.133528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.133808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.134442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.136436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.167790 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2184/4317] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:06 2022] POST /identity/v3/users => generated 309 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:06:06.171936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.186934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335f29c-553a-4134-8d70-2c82021b1f0f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.193609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.194665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.195190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.195738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.197155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.199808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.225613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2134/4318] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:06.230189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.241908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb86152-390e-4bbd-8efe-cd45fbbcd5dc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.247040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.247817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.248392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.248789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] PATH_INFO: `/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d/users/a154fac73b9a49ad83d7af6b8517a8c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.263115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:06.272064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a15f62e7d04ab297cec0cc6da52a9d, user_id=a154fac73b9a49ad83d7af6b8517a8c2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.275531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.294934 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2185/4319] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:06 2022] PUT /identity/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d/users/a154fac73b9a49ad83d7af6b8517a8c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:06.296481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.312447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b8f3-6464-41d2-946b-27df258c25d3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.353306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:06.392309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.392914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.393188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.393457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.394093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.396313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bca93-6efc-4f2a-a4e4-ae4ce378f02b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.410158 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2135/4320] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:06.412791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.425909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ae77a-abcd-4e13-8bcb-4b523014f0c1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.432577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.433418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.433741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.434568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] PATH_INFO: `/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d/users/a154fac73b9a49ad83d7af6b8517a8c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.437058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a15f62e7d04ab297cec0cc6da52a9d, user_id=a154fac73b9a49ad83d7af6b8517a8c2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.440696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.454682 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2186/4321] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:06 2022] PUT /identity/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d/users/a154fac73b9a49ad83d7af6b8517a8c2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:06.479779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.479779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.479779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.503433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] MFA Rules not processed for user `a154fac73b9a49ad83d7af6b8517a8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:06.505635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:06.600848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2136/4322] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:06.610139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:06.612401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:06.760828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56955656-bd30-449f-b895-43538b8ae4fc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:06.764634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:06.765350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:06.765722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:06.766189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:06.835221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:06.836861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:06.847908 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2187/4323] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:06 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:06:07.332519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:07.334132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:07.478848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad32f2-3f2b-4cdc-9e97-e1787eec0e7d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:07.488242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:07.489141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:07.489727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:07.490365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:07.506907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:07.510532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910c813-5c01-456b-8f31-27b7d76475a7 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:07.545356 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2137/4324] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:12.583043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:12.585367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:12.672692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f34e-04e3-42d3-9b66-71d74418c90b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:12.679099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:12.680113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:12.680113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:12.680382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] PATH_INFO: `/v3/users/0244849b631a43278742a5fd6bca5141` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:12.681378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0244849b631a43278742a5fd6bca5141)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:12.683783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:12.734104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:12.755772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0244849b631a43278742a5fd6bca5141'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:12.756563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0244849b631a43278742a5fd6bca5141'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:12.767326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0244849b631a43278742a5fd6bca5141'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:12.793528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0244849b631a43278742a5fd6bca5141'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:12.795501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0244849b631a43278742a5fd6bca5141'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:12.828668 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2188/4325] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:12 2022] DELETE /identity/v3/users/0244849b631a43278742a5fd6bca5141 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:12.833200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:12.834115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:12.834618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:12.859998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:12.861297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:12.903961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:12.956852 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2138/4326] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:12.967822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:12.969650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:13.049274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9cf3-14f7-49b5-8db7-176f2edcc7fd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:13.052500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:13.053426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:13.054336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:13.054796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:13.071697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:13.086637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:13.100144 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2189/4327] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:13.346014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:13.347888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:13.364955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e3bb-f9e6-49fe-8597-d671190db7c1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:13.375989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:13.379279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:13.379956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:13.380642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] PATH_INFO: `/v3/projects/969b9631431c4f5ebade8c3e2a55aed4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:13.382257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=969b9631431c4f5ebade8c3e2a55aed4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:13.384593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:13.467439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '969b9631431c4f5ebade8c3e2a55aed4', 'request_id': 'req-71fc4ad9-0f03-4b37-af4a-aa2691202049'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.479767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '969b9631431c4f5ebade8c3e2a55aed4', 'request_id': 'req-71fc4ad9-0f03-4b37-af4a-aa2691202049'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.495694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '969b9631431c4f5ebade8c3e2a55aed4', 'request_id': 'req-71fc4ad9-0f03-4b37-af4a-aa2691202049'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.520704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] The token cache is being invalidate because project 969b9631431c4f5ebade8c3e2a55aed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:13.521391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.522327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.524525 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2139/4328] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:13 2022] DELETE /identity/v3/projects/969b9631431c4f5ebade8c3e2a55aed4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:13.528262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:13.545062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686500-17b0-4bd9-97f8-063404033654 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:13.685859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:13.687364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:13.690186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:13.690186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] PATH_INFO: `/v3/users/c153f4354803475493e776a6473f90a6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:13.694370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c153f4354803475493e776a6473f90a6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:13.697456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:13.795023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c153f4354803475493e776a6473f90a6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.795717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c153f4354803475493e776a6473f90a6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.809214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c153f4354803475493e776a6473f90a6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.810789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:13.837203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c153f4354803475493e776a6473f90a6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.838042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4f83551-8ac2-4114-979f-e7b80305d720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c153f4354803475493e776a6473f90a6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:13.867865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2190/4329] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:13 2022] DELETE /identity/v3/users/c153f4354803475493e776a6473f90a6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:14.314435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:14.316536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:14.346446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc4ad9-0f03-4b37-af4a-aa2691202049 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:14.464251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:14.465874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:14.466478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:14.467077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] PATH_INFO: `/v3/projects/f77d3c331b7940fabb8680aba4569c02` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:14.469420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77d3c331b7940fabb8680aba4569c02)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:14.473373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:14.574320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77d3c331b7940fabb8680aba4569c02', 'request_id': 'req-e790697b-ca8c-42fb-be0c-13d6322b70a7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:14.583984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77d3c331b7940fabb8680aba4569c02', 'request_id': 'req-e790697b-ca8c-42fb-be0c-13d6322b70a7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:14.587758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:14.596289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77d3c331b7940fabb8680aba4569c02', 'request_id': 'req-e790697b-ca8c-42fb-be0c-13d6322b70a7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:14.621914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] The token cache is being invalidate because project f77d3c331b7940fabb8680aba4569c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:14.622382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:14.623286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e790697b-ca8c-42fb-be0c-13d6322b70a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:14.625111 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2140/4330] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:14 2022] DELETE /identity/v3/projects/f77d3c331b7940fabb8680aba4569c02 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:14.631841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:14.632126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:14.632589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:14.658104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:14.659884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:14.739522 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2191/4331] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:14.747801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:14.748510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:14.749069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:14.776306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:14.777552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:14.791055 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2141/4332] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:14.792677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:14.817914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448d463-97a8-4ab0-ba39-0ed34b9ecfc5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:14.827673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:14.828565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:14.829597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:14.830348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:14.832826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:14.833573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:14.836643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:14.858535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2192/4333] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:14.863644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:14.873744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f03704-d3f8-427c-9b34-186936ebd6ed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:14.879652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:14.880396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:14.880753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:14.881103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:14.881975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:14.884039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:14.934522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2142/4334] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:14 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:14.938851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:14.951998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaf101-3587-4081-aa05-2e112f5b76c6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.047641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.048974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.049479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.049769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.050458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.052917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.092956 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2193/4335] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:14 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:06:15.098665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.115935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07105-78fa-46e2-bd0d-fab684abee54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.123092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.123884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.124262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.124638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.125387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.127959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.148105 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2143/4336] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:15.151046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.180654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b284-e676-4100-b292-051dfbc9db03 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.201567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.202561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.203027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.203483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.204316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.206117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec4e53-b0a8-4751-9292-7ca1823169b2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.238703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.239030 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2194/4337] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:15.256506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f6d88-0d96-4bd3-ae67-a0de8550f19f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.277969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.278693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.278971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.279240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] PATH_INFO: `/v3/projects/5881158414ab49e98db8079b78bd6888/users/1869e3f092a9424f96c6a3f14bbad0ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.311632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5881158414ab49e98db8079b78bd6888, user_id=1869e3f092a9424f96c6a3f14bbad0ff, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.314690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.332782 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2144/4338] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:15 2022] PUT /identity/v3/projects/5881158414ab49e98db8079b78bd6888/users/1869e3f092a9424f96c6a3f14bbad0ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:15.344377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3454e-389c-4076-9043-5787888beeff None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.344802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3454e-389c-4076-9043-5787888beeff None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.344902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3454e-389c-4076-9043-5787888beeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.393280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c9c3454e-389c-4076-9043-5787888beeff None None] MFA Rules not processed for user `1869e3f092a9424f96c6a3f14bbad0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:15.404614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9c3454e-389c-4076-9043-5787888beeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:15.447611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:15.524081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2195/4339] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:15.528888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.636054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9295f-81a9-48ad-983d-499c9bc9bebc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.642226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.643121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.643378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.643642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.644326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.649367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.696682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.700786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.700786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.710836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2145/4340] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:15 2022] POST /identity/v3/projects => generated 393 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:15.711745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.754464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] MFA Rules not processed for user `370b057b12c148cdbb673ba6c06bd6b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:15.756116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:15.826797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44899da6-017a-43fb-9511-67f559c3788c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.841301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.842410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.842933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.843820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.844381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.847834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.878248 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2196/4341] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:15.893839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=1,keystone:SELECT=93,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:15.903315 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2146/4342] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:15 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:15.903529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.904507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.904819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:15.924593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bb363-8fca-4cfe-afb4-40d9c3f788da admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:15.939780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:15.940415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:15.940699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:15.940983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:15.942226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:15.946280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:15.968875 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2147/4343] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:15.973623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:15.992547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01e8e7-a1df-4433-8e59-e4b3f7349242 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:16.005830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:16.006889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.007653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.008007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] PATH_INFO: `/v3/projects/4917169139064803b4604201d67496da/users/d6989386122a4f51b3d16d4de9a21e4b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.024895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4917169139064803b4604201d67496da, user_id=d6989386122a4f51b3d16d4de9a21e4b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:16.027805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:16.040664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e591a5-6130-4980-8b48-bf3e64bc7d74 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:16.047968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2148/4344] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:15 2022] PUT /identity/v3/projects/4917169139064803b4604201d67496da/users/d6989386122a4f51b3d16d4de9a21e4b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:16.052607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:16.155502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:16.158874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a95c0-1a21-4c3e-b666-6d23c5a3a5b6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:16.178527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:16.178527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.178527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.178527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.179077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:16.179077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:16.194634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:16.195420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.196680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.197110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.210279 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2149/4345] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:16.211440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:16.239952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1111eb3-eabb-4d24-9408-b24150cdd49e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:16.263363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:16.264055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.266230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.266391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] PATH_INFO: `/v3/projects/4917169139064803b4604201d67496da/users/d6989386122a4f51b3d16d4de9a21e4b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.273215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4917169139064803b4604201d67496da, user_id=d6989386122a4f51b3d16d4de9a21e4b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:16.280410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5ea7-ceee-441f-ae95-abf59fbd87c5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:16.322269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2150/4346] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:16 2022] PUT /identity/v3/projects/4917169139064803b4604201d67496da/users/d6989386122a4f51b3d16d4de9a21e4b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:16.334164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205460-0590-4649-80e0-8305424de70a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.334417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205460-0590-4649-80e0-8305424de70a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.334990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205460-0590-4649-80e0-8305424de70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.418436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:16.421124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:16.446320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1c205460-0590-4649-80e0-8305424de70a None None] MFA Rules not processed for user `d6989386122a4f51b3d16d4de9a21e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:16.452592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c205460-0590-4649-80e0-8305424de70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:16.482726 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2197/4347] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:16.657406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2151/4348] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:16.689898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:16.691468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:16.780112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0e04-a246-4ccb-b0d2-ec1533b9c2c5 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:16.865481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:16.866955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:16.867374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:16.868082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:16.882292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:16.883984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:16.901064 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2198/4349] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:19.491191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c205460-0590-4649-80e0-8305424de70a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:19.493862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c205460-0590-4649-80e0-8305424de70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:19.615858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:19.617301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:19.623440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:19.711055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c205460-0590-4649-80e0-8305424de70a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:19.735847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:19.738884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:19.742275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:19.743140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:19.745519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:19.762352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:19.774345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:19.776985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda33e5-b725-4807-b94c-79e7922df578 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:19.792125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33007a-cedf-4570-941a-ddf5c2740232 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:19.797620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:19.798280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:19.798578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:19.798903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] PATH_INFO: `/v3/users/66ffbeeec3274f47a826fcc5e541feaa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:19.800097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ffbeeec3274f47a826fcc5e541feaa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:19.802641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:19.839203 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2152/4350] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:19.928519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ffbeeec3274f47a826fcc5e541feaa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:19.932818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ffbeeec3274f47a826fcc5e541feaa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:19.948221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ffbeeec3274f47a826fcc5e541feaa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:19.973712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ffbeeec3274f47a826fcc5e541feaa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:19.974700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ffbeeec3274f47a826fcc5e541feaa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:19.993068 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2199/4351] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:19 2022] DELETE /identity/v3/users/66ffbeeec3274f47a826fcc5e541feaa => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:19.997632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:19.997951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:19.998304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:20.025860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:20.027609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:20.225867 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2153/4352] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:20.248040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:20.249595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:20.383502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4c64-865d-4065-8a88-a362d01cdcba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:20.390114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:20.393288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:20.393573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:20.394579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:20.418338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:20.437644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:20.477484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2200/4353] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:20.859743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:20.861301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:20.878958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc51ca-c766-4d26-a3c5-f9056dd979b0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:20.896608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:20.897271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:20.897621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:20.897836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] PATH_INFO: `/v3/projects/2ae36fb9a68c48fc9a7888a71146725a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:20.899197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae36fb9a68c48fc9a7888a71146725a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:20.901136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:20.973071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae36fb9a68c48fc9a7888a71146725a', 'request_id': 'req-18dfbc18-b8de-4bed-a2cb-7613698d0a09'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:20.979494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae36fb9a68c48fc9a7888a71146725a', 'request_id': 'req-18dfbc18-b8de-4bed-a2cb-7613698d0a09'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:20.988139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae36fb9a68c48fc9a7888a71146725a', 'request_id': 'req-18dfbc18-b8de-4bed-a2cb-7613698d0a09'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.003669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] The token cache is being invalidate because project 2ae36fb9a68c48fc9a7888a71146725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:21.003968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.004848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-18dfbc18-b8de-4bed-a2cb-7613698d0a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.006658 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2154/4354] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:20 2022] DELETE /identity/v3/projects/2ae36fb9a68c48fc9a7888a71146725a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:21.010894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:21.022312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9267a14-0603-4694-8640-1f775d16499d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:21.030079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:21.030653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:21.031136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:21.077746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] MFA Rules not processed for user `370b057b12c148cdbb673ba6c06bd6b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:21.079462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:21.115311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:21.115973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:21.116382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:21.116676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] PATH_INFO: `/v3/users/a154fac73b9a49ad83d7af6b8517a8c2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:21.117823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a154fac73b9a49ad83d7af6b8517a8c2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:21.119911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:21.126829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:21.139164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:21.159609 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2155/4355] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:21.179578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a154fac73b9a49ad83d7af6b8517a8c2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.179807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a154fac73b9a49ad83d7af6b8517a8c2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.190028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a154fac73b9a49ad83d7af6b8517a8c2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.215448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a154fac73b9a49ad83d7af6b8517a8c2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.216396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a154fac73b9a49ad83d7af6b8517a8c2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:21.254574 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2201/4356] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:21 2022] DELETE /identity/v3/users/a154fac73b9a49ad83d7af6b8517a8c2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:21.706004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:21.708244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:21.776746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cc8da-8b39-480d-96ca-d0c4138dd477 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:21.835919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:21.836633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:21.836911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:21.837180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:21.839221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:21.853861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:21.853861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:21.855698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:21.857588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:21.891322 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2156/4357] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:21.963487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ac89a-ef01-4e5c-b383-909cc0a23283 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:21.970148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:21.970920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:21.971222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:21.971467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] PATH_INFO: `/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:21.973472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a15f62e7d04ab297cec0cc6da52a9d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:21.976247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.073084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a15f62e7d04ab297cec0cc6da52a9d', 'request_id': 'req-4cd29448-6151-4865-a209-f4a2d0d25675'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.079088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a15f62e7d04ab297cec0cc6da52a9d', 'request_id': 'req-4cd29448-6151-4865-a209-f4a2d0d25675'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.082070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.084405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.087683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a15f62e7d04ab297cec0cc6da52a9d', 'request_id': 'req-4cd29448-6151-4865-a209-f4a2d0d25675'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.105438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] The token cache is being invalidate because project 66a15f62e7d04ab297cec0cc6da52a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:22.105762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9658-9f02-4f80-9fd2-20780b4e7920 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.105957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.106729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cd29448-6151-4865-a209-f4a2d0d25675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.108517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2202/4358] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:21 2022] DELETE /identity/v3/projects/66a15f62e7d04ab297cec0cc6da52a9d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:22.113505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a155f-790c-41e5-b981-83900022a8ce None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.113793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a155f-790c-41e5-b981-83900022a8ce None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.114075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a155f-790c-41e5-b981-83900022a8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.138460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2d7a155f-790c-41e5-b981-83900022a8ce None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:22.139904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2d7a155f-790c-41e5-b981-83900022a8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.161313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:22.181723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.183028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.183314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.183592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] PATH_INFO: `/v3/users/1869e3f092a9424f96c6a3f14bbad0ff` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.184785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1869e3f092a9424f96c6a3f14bbad0ff)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.186839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.211373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2203/4359] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:22.221601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.221890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.222188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.248064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1869e3f092a9424f96c6a3f14bbad0ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.248490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:22.248742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1869e3f092a9424f96c6a3f14bbad0ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.250098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.252958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:22.259717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1869e3f092a9424f96c6a3f14bbad0ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.263009 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2204/4360] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:22.267112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.273419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1869e3f092a9424f96c6a3f14bbad0ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.275006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5e32d64-0a75-430e-978d-04604b35a335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1869e3f092a9424f96c6a3f14bbad0ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:22.278311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f529798-2ba4-4a31-bc1c-6ad1e8cafa31 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.284574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.285952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.285952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.286231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.286903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:22.287203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.288797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.304755 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2205/4361] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:06:22.308880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.313908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2157/4362] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:22 2022] DELETE /identity/v3/users/1869e3f092a9424f96c6a3f14bbad0ff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:22.318552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.319646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.319856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.323675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea91d53-99af-496d-8047-58fc9d3cc83a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.348760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:22.350324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.397036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.397957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.398817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.399197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.400019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.402786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.432411 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2158/4363] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:22.438377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2206/4364] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:06:22.443100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.443390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.444937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.455162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4f4e6-b376-4bdc-9556-fcca207e3e3e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.520612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e283-e718-4649-b5a5-96f4657fede8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.525135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.526005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.526385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.526698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.527710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.528370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.528581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.528853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.529750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.531892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.539409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.541174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.555412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2207/4365] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:22.559106 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2159/4366] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:22.563597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.574354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd515880-0efb-4931-9cb1-f3f7178b44fe service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.579132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.579959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.580158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.580439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.581064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.582939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.594986 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2208/4367] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:22.600568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.611952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e818-bdf7-49b7-a70a-6f94b6c796a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.618570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.619779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.619971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.620234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.621543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.623665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69ca4f-66d7-4b06-9d10-27fce1789a9a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.640617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2160/4368] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:22.653889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:22.671215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ebe62-3e5c-4c47-8b7b-09507f29e24b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:22.685083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:22.685765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.686043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.686317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] PATH_INFO: `/v3/projects/3fbcb4a735d84e789094154184e8d622/users/cddcfb5cd4c74808b09a1ca4358fc4c1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.719855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fbcb4a735d84e789094154184e8d622, user_id=cddcfb5cd4c74808b09a1ca4358fc4c1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:22.724103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:22.727980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:22.747128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2209/4369] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:22 2022] PUT /identity/v3/projects/3fbcb4a735d84e789094154184e8d622/users/cddcfb5cd4c74808b09a1ca4358fc4c1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:22.756781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:22.757073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:22.757348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:22.799241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] MFA Rules not processed for user `cddcfb5cd4c74808b09a1ca4358fc4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:22.800621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:22.890216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:22.928362 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2161/4370] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:22.929002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.005774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.007249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:23.051947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdec8d2-6260-48a6-a4d2-dc5634da7bbe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.059647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.061612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.063105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.063651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.064512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.068626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.118165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2210/4371] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:22 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:23.119828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.156932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f0888-e306-433b-8fee-4e41b66da914 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.224226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244a9cb-ef19-4851-a4c2-73efd8ba5f91 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.229374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.230215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.230361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.230557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.231197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.232157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.232913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.233263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.233727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] PATH_INFO: `/v3/projects/5881158414ab49e98db8079b78bd6888` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.235327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=5881158414ab49e98db8079b78bd6888)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.235479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.237578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.276723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2211/4372] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:23 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:23.280155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.292613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01029ad-a4b0-4b1e-80ca-04832eeb9db7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.319633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5881158414ab49e98db8079b78bd6888', 'request_id': 'req-2389915c-c0ee-499a-a976-37f5730a9ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.333219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5881158414ab49e98db8079b78bd6888', 'request_id': 'req-2389915c-c0ee-499a-a976-37f5730a9ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.352320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5881158414ab49e98db8079b78bd6888', 'request_id': 'req-2389915c-c0ee-499a-a976-37f5730a9ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.372830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] The token cache is being invalidate because project 5881158414ab49e98db8079b78bd6888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:23.373228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.374597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.375360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.375981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.376261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.376534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.380031 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2162/4373] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:23 2022] DELETE /identity/v3/projects/5881158414ab49e98db8079b78bd6888 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:23.380213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.380213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.381359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.383962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:23.393630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2212/4374] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:23.399328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.401543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:23.403546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389915c-c0ee-499a-a976-37f5730a9ece admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.416180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.416825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.417105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.417373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] PATH_INFO: `/v3/users/d6989386122a4f51b3d16d4de9a21e4b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.419489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6989386122a4f51b3d16d4de9a21e4b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.420773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621831e-b56c-4a12-8698-9adfbb362306 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.422136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.426622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.427401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.427666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.427931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] PATH_INFO: `/v3/projects/129c4ea8234040c5883fd66abcd5fcbe/users/4653229d038e4ee2a9488c82a7ae87c8/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.430488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:23.444790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:23.451988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=129c4ea8234040c5883fd66abcd5fcbe, user_id=4653229d038e4ee2a9488c82a7ae87c8, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.458675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.482419 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2213/4375] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:23 2022] PUT /identity/v3/projects/129c4ea8234040c5883fd66abcd5fcbe/users/4653229d038e4ee2a9488c82a7ae87c8/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:23.483786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.508482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6989386122a4f51b3d16d4de9a21e4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.509422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6989386122a4f51b3d16d4de9a21e4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.519555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6989386122a4f51b3d16d4de9a21e4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.533222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6989386122a4f51b3d16d4de9a21e4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.536155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6989386122a4f51b3d16d4de9a21e4b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:23.564762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2163/4376] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:23 2022] DELETE /identity/v3/users/d6989386122a4f51b3d16d4de9a21e4b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:23.623400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57499d24-a4d9-46ff-ace1-523bf533c9fb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.744731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.745486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.745841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.746226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.746984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.748762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.768293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.770685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:23.775178 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2214/4377] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:23.791607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1349fd-5545-4a6b-8872-d2336b52e8d9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.799391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:23.800135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.801241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.802458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] PATH_INFO: `/v3/projects/129c4ea8234040c5883fd66abcd5fcbe/users/4653229d038e4ee2a9488c82a7ae87c8/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.805619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129c4ea8234040c5883fd66abcd5fcbe, user_id=4653229d038e4ee2a9488c82a7ae87c8, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:23.809693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e571c0-61c4-4270-b163-7d92b6dba180 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:23.824498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2164/4378] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:23 2022] PUT /identity/v3/projects/129c4ea8234040c5883fd66abcd5fcbe/users/4653229d038e4ee2a9488c82a7ae87c8/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:23.835347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:23.837491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:23.850308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:23.850619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:23.851039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:23.857459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407323fa-cc40-4777-9230-19599aa5b2f2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:23.897665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] MFA Rules not processed for user `4653229d038e4ee2a9488c82a7ae87c8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:23.899108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:24.025111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.027065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.027429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.027774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] PATH_INFO: `/v3/projects/4917169139064803b4604201d67496da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.030487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4917169139064803b4604201d67496da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.034260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.100308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2165/4379] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:24.108051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:24.111438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.113437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:24.145107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4917169139064803b4604201d67496da', 'request_id': 'req-2c494920-f78a-47f3-a773-e07b89379ddb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:24.156141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4917169139064803b4604201d67496da', 'request_id': 'req-2c494920-f78a-47f3-a773-e07b89379ddb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:24.169525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4917169139064803b4604201d67496da', 'request_id': 'req-2c494920-f78a-47f3-a773-e07b89379ddb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:24.192923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] The token cache is being invalidate because project 4917169139064803b4604201d67496da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:24.193323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:24.194158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c494920-f78a-47f3-a773-e07b89379ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:24.196373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2215/4380] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:23 2022] DELETE /identity/v3/projects/4917169139064803b4604201d67496da => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:24.218401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94e89b-bc79-44d5-a02a-0156e04c62b7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.218825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94e89b-bc79-44d5-a02a-0156e04c62b7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.219203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94e89b-bc79-44d5-a02a-0156e04c62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.247641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1c94e89b-bc79-44d5-a02a-0156e04c62b7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:24.249231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1c94e89b-bc79-44d5-a02a-0156e04c62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:24.265146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb758aa-b852-4c5f-b9cc-251347317270 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.293013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:24.356882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.357722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.358161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.358566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.360959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:24.394596 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2216/4381] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:24.396252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.396549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.396873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.427450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:24.429235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:24.438722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2217/4382] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:24.443754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.454420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce415a5-71c0-4a05-9f91-cf8dc4933c06 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.462915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.463735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.464392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.464685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.465119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:24.465409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.468622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.492492 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2218/4383] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:24.496922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.498287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.499965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.516337 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2166/4384] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:24.521543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e8a94-45a7-4242-b5f2-65159b1e178b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.528898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.529732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.530339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.530815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.531633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.534312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.574533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2219/4385] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:24.579592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.596580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec9a2a-411e-4751-bad2-988487f03dcb service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.704424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.705806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.707924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.708488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.709364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.712985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.744839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2167/4386] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:24 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:24.746130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.760047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90222ccb-800b-418e-b8f9-485ecd25e24c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.767129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.767881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.768265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.768640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.770380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.772274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.787707 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2220/4387] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:24.791444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.804738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc24cc-6f2a-442a-ac97-761c8c671348 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.811993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.812985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.813254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.813523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.814167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.816388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b83d33-8b98-48ac-aaee-268de96cabc9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.831198 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2168/4388] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:24.833210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:24.850930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db4f8d-ae4b-4ba0-bb84-203d43165845 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:24.861133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:24.862763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.863293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.864077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] PATH_INFO: `/v3/projects/6ce96a6207c24743acb3576dd26f0497/users/a2531da6263c45e7a700b183482df05f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.893086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce96a6207c24743acb3576dd26f0497, user_id=a2531da6263c45e7a700b183482df05f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:24.899926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:24.918911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2221/4389] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:24 2022] PUT /identity/v3/projects/6ce96a6207c24743acb3576dd26f0497/users/a2531da6263c45e7a700b183482df05f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:24.932593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:24.932882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:24.933154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:24.967796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] MFA Rules not processed for user `a2531da6263c45e7a700b183482df05f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:24.969037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:25.031913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:25.075149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2169/4390] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:25.127147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:25.129684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:25.193909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:25.225460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebd372-b922-40f0-892c-04f31f019e68 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:25.235257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:25.237052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:25.237556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:25.238694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:25.266703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:25.268307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:25.287326 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2222/4391] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:28.561461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:28.564553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:28.714775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f29030-e29c-4256-90d9-c1c39b7b9054 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:28.726709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:28.728402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:28.728763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:28.729157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] PATH_INFO: `/v3/users/762f4e35de9e4083bd0e7a4d0de3c136` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:28.731525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] RBAC: Authorizing `identity:delete_user(user_id=762f4e35de9e4083bd0e7a4d0de3c136)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:28.733584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:28.798182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762f4e35de9e4083bd0e7a4d0de3c136'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:28.798182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762f4e35de9e4083bd0e7a4d0de3c136'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:28.808478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762f4e35de9e4083bd0e7a4d0de3c136'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:28.819041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762f4e35de9e4083bd0e7a4d0de3c136'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:28.819952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd556216-ee47-429b-b47a-9b32f5dc89db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762f4e35de9e4083bd0e7a4d0de3c136'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:28.854156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2170/4392] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:28 2022] DELETE /identity/v3/users/762f4e35de9e4083bd0e7a4d0de3c136 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:29.367806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:29.383236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7aa4c-6a7e-4725-bf55-8a8ce51e7b81 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:29.539031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:29.539981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:29.540265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:29.540663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] PATH_INFO: `/v3/projects/648cf4f432a3400b8dcf2b57a99d082e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:29.542460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=648cf4f432a3400b8dcf2b57a99d082e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:29.545182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:29.605887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648cf4f432a3400b8dcf2b57a99d082e', 'request_id': 'req-f72a297a-30bb-4a92-b018-38dedbf69db1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:29.613568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648cf4f432a3400b8dcf2b57a99d082e', 'request_id': 'req-f72a297a-30bb-4a92-b018-38dedbf69db1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:29.621721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648cf4f432a3400b8dcf2b57a99d082e', 'request_id': 'req-f72a297a-30bb-4a92-b018-38dedbf69db1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:29.649232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] The token cache is being invalidate because project 648cf4f432a3400b8dcf2b57a99d082e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:29.649232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:29.650087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f72a297a-30bb-4a92-b018-38dedbf69db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:29.654454 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2223/4393] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:29 2022] DELETE /identity/v3/projects/648cf4f432a3400b8dcf2b57a99d082e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:29.656441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e7a7-25cb-4929-92a5-ea529392f545 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:29.656902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e7a7-25cb-4929-92a5-ea529392f545 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:29.657005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818e7a7-25cb-4929-92a5-ea529392f545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:29.681287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5818e7a7-25cb-4929-92a5-ea529392f545 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:29.682938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5818e7a7-25cb-4929-92a5-ea529392f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:29.814057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:29.828201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:29.828554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:29.828756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:29.839268 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2171/4394] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:29.848413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:29.848851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:29.850173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:29.866456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:29.868198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:29.953764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2224/4395] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:29.957641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:29.961668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:29.994712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2172/4396] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:29.998751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.019942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa56201-87db-4913-901d-edf9a0de16cf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.027038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.028071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.029417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.029866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.030769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:30.031538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.033703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.072959 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2225/4397] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:30.078075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.091246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edece-5a3f-4ad0-b163-1ca4af7e914e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.106114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.108737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.109264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.109799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.110682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.114389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.166601 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2173/4398] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:30 2022] POST /identity/v3/projects => generated 397 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:30.173339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.184266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:30.190012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1f0c4-b785-4992-ae30-548df23c7408 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.273163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.273942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.274328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.274880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.275937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.278132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.309422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2226/4399] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:30 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:30.314427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.329331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6635ff6-e6e6-4e57-83a4-abc85e72c8fe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.344262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.345334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.345789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.346242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.347764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.349949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.361993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2174/4400] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:30.365828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.378749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8132f6-fcb0-4146-92a1-0536489eee11 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.389986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.391237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.391746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.392245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.393112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.397305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03be5df-688c-4775-9640-dbf77bceaca7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.418559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2227/4401] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:30.422758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.435385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc4922-f1cf-49a4-8097-b74790c0c663 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.445914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.447275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.447744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.448029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] PATH_INFO: `/v3/projects/b0c1b7c4c6ff480c82a1f9708b7dbe62/users/d142c33eccd048bfa3415b1661ee373f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.481357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c1b7c4c6ff480c82a1f9708b7dbe62, user_id=d142c33eccd048bfa3415b1661ee373f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.484396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.515748 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2175/4402] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:30 2022] PUT /identity/v3/projects/b0c1b7c4c6ff480c82a1f9708b7dbe62/users/d142c33eccd048bfa3415b1661ee373f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:06:30.517106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.517461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.517771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.567709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:30.569452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:30.695852 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2228/4403] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:30.739936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:30.742620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:30.834797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08205eae-de55-4fef-b4f4-7d479cd65a76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:30.838847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:30.839697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:30.840256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:30.840689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:30.852132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:30.854674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a58822-14af-44d5-9951-cc2957b12abb service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:30.866787 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2176/4404] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:35.020237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:35.021729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:35.034838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabab9a-cbcc-43f8-952b-99f64db87e8c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:35.040324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:35.041506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:35.041950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:35.042372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] PATH_INFO: `/v3/users/a2531da6263c45e7a700b183482df05f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:35.043642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2531da6263c45e7a700b183482df05f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:35.045966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:35.129020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2531da6263c45e7a700b183482df05f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.129814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2531da6263c45e7a700b183482df05f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.148908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2531da6263c45e7a700b183482df05f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.185155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2531da6263c45e7a700b183482df05f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.185981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2531da6263c45e7a700b183482df05f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.193820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:35.204029 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2229/4405] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:35 2022] DELETE /identity/v3/users/a2531da6263c45e7a700b183482df05f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:35.208605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:35.209368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:35.209778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:35.247203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:35.249014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:35.361838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2177/4406] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:35.374091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:35.378944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:35.490895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9552d8b-585e-4c17-8fa0-eb6716cd197a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:35.500777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:35.501477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:35.502081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:35.502367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:35.516747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:35.518349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f07b1-ef81-453f-9e8b-15c580ab6f07 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:35.535103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2230/4407] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:35.791758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:35.793463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:35.807388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01ab2-c081-47b7-bc7c-5d1a2cf0f84e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:35.824975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:35.825734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:35.826018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:35.826296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] PATH_INFO: `/v3/projects/6ce96a6207c24743acb3576dd26f0497` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:35.827663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce96a6207c24743acb3576dd26f0497)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:35.829570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:35.865225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:35.897241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce96a6207c24743acb3576dd26f0497', 'request_id': 'req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.904753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce96a6207c24743acb3576dd26f0497', 'request_id': 'req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.915634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce96a6207c24743acb3576dd26f0497', 'request_id': 'req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.928583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] The token cache is being invalidate because project 6ce96a6207c24743acb3576dd26f0497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:35.928899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.929721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33f6b489-fa3a-40cb-80f0-b3cb5c6310d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:35.931239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2178/4408] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:35 2022] DELETE /identity/v3/projects/6ce96a6207c24743acb3576dd26f0497 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:35.936740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:35.937070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:35.937450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:35.965057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:35.966472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:36.041985 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2231/4409] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:36.050753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.051032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.051306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.075000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:36.076304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:36.089214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2179/4410] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:36.093994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.106755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e790486-00a9-4508-a2b1-d1b22174a55d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.113112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.113704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.113988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.114314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.114993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:36.115295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.122873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.152464 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2232/4411] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:36.156892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.177582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929bdc-8ac1-4b3c-9b2d-7e3891179bdc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.185882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.187127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.187576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.187982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.188798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.191887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.243553 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2180/4412] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:36 2022] POST /identity/v3/projects => generated 363 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:36.247573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.262758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6686e5-4770-49da-ae0c-bebf2b3d7d60 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.314860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.316913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:36.442539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.443074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.443352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.443619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.444357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.446212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.455439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba7b84-ec89-426c-a7a7-c522ac05c895 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.468392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.470130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.470590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.474070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] PATH_INFO: `/v3/users/6910dfaa89b04d3b9316992d78e9424c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.475974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6910dfaa89b04d3b9316992d78e9424c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.478987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.526538 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2233/4413] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:36 2022] POST /identity/v3/users => generated 282 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:36.530333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.552401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bf0d7-79eb-4c64-805e-34bdf3d55f90 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.564799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.565844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.566216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.566739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.567569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.569203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.597969 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2234/4414] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:36.602411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.609668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:36.618601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e3e73-e4ba-4cce-8396-ebb2c954c992 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.627471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.633899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.633899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.633899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.633899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.633899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.666735 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2235/4415] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:06:36.668815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:36.679886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6910dfaa89b04d3b9316992d78e9424c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:36.684009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6910dfaa89b04d3b9316992d78e9424c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:36.688703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ec1e-286d-4222-8478-942eff7105d9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:36.694278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6910dfaa89b04d3b9316992d78e9424c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:36.703310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:36.703996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.704276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.704547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] PATH_INFO: `/v3/projects/de1b6b1ac46840199aa78debc44fa00d/users/0fac2afeb1aa43b980be965a1f23f12a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.709364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6910dfaa89b04d3b9316992d78e9424c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:36.710439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4777d90-7d84-49b0-89da-7d79b105ac72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6910dfaa89b04d3b9316992d78e9424c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:36.751837 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2181/4416] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:06:36 2022] DELETE /identity/v3/users/6910dfaa89b04d3b9316992d78e9424c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:36.752392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1b6b1ac46840199aa78debc44fa00d, user_id=0fac2afeb1aa43b980be965a1f23f12a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:36.754110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.754401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.754785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.755705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d98cb-c966-4276-b23f-ab53a675aaaa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:36.781999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:36.786728 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2236/4417] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:36 2022] PUT /identity/v3/projects/de1b6b1ac46840199aa78debc44fa00d/users/0fac2afeb1aa43b980be965a1f23f12a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:36.791830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:36.794191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:36.892629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:36.893063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:36.893343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:36.963639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] MFA Rules not processed for user `0fac2afeb1aa43b980be965a1f23f12a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:36.964822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:36.987144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2182/4418] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:37.111128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2237/4419] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:37.148643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:37.156963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:37.157236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:37.159167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:37.281074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1ed2e-a250-4867-b22e-931b802acea9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:37.298194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:37.298956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:37.301074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:37.301328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:37.327763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:37.329279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:37.334636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd7ebc-b7a6-4b6a-bb12-98387831751d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:37.343271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:37.344016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:37.344666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:37.345058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:37.351854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2238/4420] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:06:37.366548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:37.373244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52150-e546-4c41-87fe-b93b00264e28 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:37.401303 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2183/4421] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:38.757107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:38.758901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:38.783193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5abd9c-6dad-43a4-be3b-832594eb48ed service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:38.793350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:38.794868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:38.795415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:38.795907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] PATH_INFO: `/v3/projects/b9bb8e91bd434e1e91c19ba084855d7c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:38.798935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bb8e91bd434e1e91c19ba084855d7c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:38.802748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:38.955520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bb8e91bd434e1e91c19ba084855d7c', 'request_id': 'req-44b7f90d-af14-471c-9ddd-3723ccf3b72b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:38.966330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:38.968769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bb8e91bd434e1e91c19ba084855d7c', 'request_id': 'req-44b7f90d-af14-471c-9ddd-3723ccf3b72b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:38.983673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bb8e91bd434e1e91c19ba084855d7c', 'request_id': 'req-44b7f90d-af14-471c-9ddd-3723ccf3b72b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:39.012314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] The token cache is being invalidate because project b9bb8e91bd434e1e91c19ba084855d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:39.013694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:39.013694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-44b7f90d-af14-471c-9ddd-3723ccf3b72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:39.019407 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2239/4422] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:38 2022] DELETE /identity/v3/projects/b9bb8e91bd434e1e91c19ba084855d7c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:39.020315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.020882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.020976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.042085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:39.043644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:39.121043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2184/4423] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:06:39.122845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.123154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.123427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.146191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:39.147595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:39.157135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2240/4424] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:39.162411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.175008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90061bbe-6f08-485a-bca3-31ce7facb1b5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.180442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.181068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.181346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.181746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.182283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:39.182633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.184433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.200728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2185/4425] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:39.205505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.216384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677180cd-3c16-48ee-a8ff-b12cccc8abf7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.221453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.222169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.222449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.222862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.223505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.225513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.261666 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2241/4426] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:39.266659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.285015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235e9c-8839-40ed-9857-8e5c079def10 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.336565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:39.365196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.365814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.366091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.366359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.367284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.369633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.418758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.419943 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2186/4427] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:06:39.436594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c1b51-c22a-4d68-ace8-781cd3f2dde4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.442158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.442819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.443098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.443368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.444991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.446187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.460089 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2242/4428] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:39.463970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.479813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295d46b-9898-4787-b7de-ac03817dbb91 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.496407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.497233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.498893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.499797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.501146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.503747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.531296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.533631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:39.570208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2187/4429] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:06:39.572465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.603508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7482-993b-4290-b3a7-cdd354783442 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.626733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.627419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.632764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.633073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] PATH_INFO: `/v3/projects/d0e5867b34544cadbb354439808248aa/users/9eca15a31e064f4199bfc4b04de7c5b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.692923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e5867b34544cadbb354439808248aa, user_id=9eca15a31e064f4199bfc4b04de7c5b2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.701898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a9c39-e2ac-4063-9d91-8b3d0df5d006 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.744976 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2188/4430] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:06:39 2022] PUT /identity/v3/projects/d0e5867b34544cadbb354439808248aa/users/9eca15a31e064f4199bfc4b04de7c5b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:39.750844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.751130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.751408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.775917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7b99-43ff-4bad-a2a0-62782c6fc6e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.794455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:39.795772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:39.903314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2189/4431] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:39.904001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4653229d038e4ee2a9488c82a7ae87c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129c4ea8234040c5883fd66abcd5fcbe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.904614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.904938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.905222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.906898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:39.906898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.908134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:39.922631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2243/4432] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:06:39.927338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.933671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:39.941391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:39.950889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a5c96-e246-4ce6-ac67-97b65c069618 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:39.957097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4653229d038e4ee2a9488c82a7ae87c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129c4ea8234040c5883fd66abcd5fcbe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:39.957731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:39.958236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:39.958558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:39.959380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:39.961555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:40.011202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2190/4433] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:06:39 2022] POST /identity/v3/projects => generated 348 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:06:40.022798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:40.066489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:40.102607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bd5b6-0f2f-4592-b9a6-402eecfe26b1 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:40.157357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d4c5a-0558-4504-822c-39bce9d224d0 tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:40.165906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4653229d038e4ee2a9488c82a7ae87c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129c4ea8234040c5883fd66abcd5fcbe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:40.167454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:40.167454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:40.167454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:40.170826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:06:40.171183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:40.173093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:40.197469 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2191/4434] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:06:40.212783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:40.213933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:40.214427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:40.214890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:40.317707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:40.320249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:40.344441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2244/4435] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:42.180291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:42.192343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:42.198055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8a1f-11f1-4546-ae79-82c158fd80ed tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:42.221631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4653229d038e4ee2a9488c82a7ae87c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '129c4ea8234040c5883fd66abcd5fcbe', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:42.224215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:42.225735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:42.226615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] PATH_INFO: `/v3/projects/47e90dee14584b2f9e133aa7e378b1b3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:42.230275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorizing `identity:delete_project(project_id=47e90dee14584b2f9e133aa7e378b1b3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:42.263768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:42.402479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e90dee14584b2f9e133aa7e378b1b3', 'request_id': 'req-9646c1dc-584d-47a9-8a97-ce0354c19acc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:42.416698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e90dee14584b2f9e133aa7e378b1b3', 'request_id': 'req-9646c1dc-584d-47a9-8a97-ce0354c19acc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:42.441084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e90dee14584b2f9e133aa7e378b1b3', 'request_id': 'req-9646c1dc-584d-47a9-8a97-ce0354c19acc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:42.488457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] The token cache is being invalidate because project 47e90dee14584b2f9e133aa7e378b1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:06:42.490711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:42.495168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9646c1dc-584d-47a9-8a97-ce0354c19acc tempest-RoutersAdminTest-1094273110 tempest-RoutersAdminTest-1094273110-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:06:42.501596 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2192/4436] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:06:42 2022] DELETE /identity/v3/projects/47e90dee14584b2f9e133aa7e378b1b3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:06:42.579660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:42.598112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29437d5c-61a8-416d-88e7-7a31c9f688eb service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:42.731362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:42.732340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:42.732870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:42.733332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:42.736419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:42.933197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:42.938273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a56cb-69d3-4bb5-abdb-91a72f83f58f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:42.959013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2245/4437] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:49.572468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:49.573306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:49.573717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:49.638549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] MFA Rules not processed for user `d142c33eccd048bfa3415b1661ee373f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:49.641860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:49.812727 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2193/4438] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:49.823248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ba09-ece3-4ad8-b806-37e8a3614029 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:49.823822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ba09-ece3-4ad8-b806-37e8a3614029 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:49.824029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9ba09-ece3-4ad8-b806-37e8a3614029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:49.861254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:49.891305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-86f9ba09-ece3-4ad8-b806-37e8a3614029 None None] MFA Rules not processed for user `d142c33eccd048bfa3415b1661ee373f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:49.892864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86f9ba09-ece3-4ad8-b806-37e8a3614029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:49.911725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2246/4439] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:49.923436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:49.926267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:50.074405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78446c-f792-469c-93ca-3c31b1de05c8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:50.102855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:50.104181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:50.104683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:50.105457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:50.132899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:50.138082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:50.195314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2194/4440] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:52.023596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574abec-a929-4992-a272-ab5b285d4c6d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:52.024137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574abec-a929-4992-a272-ab5b285d4c6d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:52.024336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574abec-a929-4992-a272-ab5b285d4c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:52.108655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6574abec-a929-4992-a272-ab5b285d4c6d None None] MFA Rules not processed for user `9eca15a31e064f4199bfc4b04de7c5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:52.112394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6574abec-a929-4992-a272-ab5b285d4c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:52.226550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2247/4441] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:06:52.551665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:06:52.562783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:06:52.576327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb0e54-91ab-4312-8eb8-1d4f5f296d57 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:06:52.591408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:06:52.592637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:52.593761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:52.594295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:52.611213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:06:52.615067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100105f0-df24-43fe-b7e7-740ffd69ad9a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:06:52.663140 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2195/4442] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:06:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:06:57.274517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:06:57.275161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:06:57.275161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:06:57.315262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] MFA Rules not processed for user `9eca15a31e064f4199bfc4b04de7c5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:06:57.317251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:06:57.340524 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2248/4443] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:02.647866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:05.707373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:05.708064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:05.708064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:05.727951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] MFA Rules not processed for user `6493da825aa24f43bdf8f0835a16e3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:05.729502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:05.811596 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2196/4444] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:07.302070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=35 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:10.309403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:10.310937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:10.459573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607b941-9ffd-4b95-844b-61a1e3879158 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:10.467133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:10.477893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:10.479132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:10.479578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] PATH_INFO: `/v3/users/e557d91b49c143499d809a55e85932fd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:10.481313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e557d91b49c143499d809a55e85932fd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:10.484440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:10.545543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e557d91b49c143499d809a55e85932fd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:10.546410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e557d91b49c143499d809a55e85932fd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:10.560384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e557d91b49c143499d809a55e85932fd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:10.580854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e557d91b49c143499d809a55e85932fd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:10.583357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b87c7dcc-a62f-4067-9888-1739aa79215b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e557d91b49c143499d809a55e85932fd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:10.615517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2249/4445] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:07:10 2022] DELETE /identity/v3/users/e557d91b49c143499d809a55e85932fd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:11.434625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:11.453505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20cd71-6298-4f7f-bb81-5d7b9c38135e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:11.559879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:11.560751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:11.561554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:11.562077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] PATH_INFO: `/v3/projects/accafa2724354ca8b1c9f19940989429` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:11.564900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=accafa2724354ca8b1c9f19940989429)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:11.568187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:11.679044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'accafa2724354ca8b1c9f19940989429', 'request_id': 'req-3ada5dde-8da4-498e-862e-bb7e078ce4bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:11.685655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'accafa2724354ca8b1c9f19940989429', 'request_id': 'req-3ada5dde-8da4-498e-862e-bb7e078ce4bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:11.700655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'accafa2724354ca8b1c9f19940989429', 'request_id': 'req-3ada5dde-8da4-498e-862e-bb7e078ce4bf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:11.729186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] The token cache is being invalidate because project accafa2724354ca8b1c9f19940989429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:07:11.730199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:11.731628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3ada5dde-8da4-498e-862e-bb7e078ce4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:11.734411 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2197/4446] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:07:11 2022] DELETE /identity/v3/projects/accafa2724354ca8b1c9f19940989429 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:11.740415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:11.740946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:11.740946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:11.763336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:11.764660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:11.855367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:11.855651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:11.856185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:11.860207 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2250/4447] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:11.882147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:11.883634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:11.894955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2198/4448] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:11.899204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:11.911119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89289b4-891c-40d7-a2b3-472ddde4c1f8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:11.916624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:11.917255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:11.917532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:11.917800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:11.918452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:07:11.918884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:11.920488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:11.934278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2251/4449] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:07:11.938591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:11.959117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbbeb3-842c-4cb8-8a82-dfa64fa04412 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:11.968508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:11.969570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:11.971320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:11.971808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:11.972908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:11.975674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:12.022385 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2199/4450] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:07:11 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:07:12.026461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:12.039919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4c02-ba16-4ee0-a27c-c88b30ae440c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:12.044249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:12.186252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:12.187143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.188204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.188622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.190933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:12.193809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:12.232767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2252/4451] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:07:12 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:07:12.234905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:12.248698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87cc1-f26a-4e1a-86ec-c2d1b64f3923 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:12.254056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:12.254825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.255154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.255541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.256542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:12.258372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:12.260809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:12.280895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2200/4452] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:07:12.282133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:12.297435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf87249-e77d-46dd-ab11-b7bfb48754be admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:12.302893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:12.303507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.303936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.304214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.305102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:12.307009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3d31b-7782-4b3e-94d4-f6d08ed72808 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:12.348110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2253/4453] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:07:12.352228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:12.371143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb498b7d-abba-47dc-a174-c7ce76939795 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:12.396903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:12.400327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.403801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.406009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] PATH_INFO: `/v3/projects/28e486847ec742fbbd1ed13b404eb519/users/f718a9c5980641d3b67ce061de573c71/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.457023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e486847ec742fbbd1ed13b404eb519, user_id=f718a9c5980641d3b67ce061de573c71, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:12.460774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4cdb-d393-4ec0-ad57-66176621b7ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:12.506530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2201/4454] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:07:12 2022] PUT /identity/v3/projects/28e486847ec742fbbd1ed13b404eb519/users/f718a9c5980641d3b67ce061de573c71/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:12.585195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.585657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.586025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.669457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] MFA Rules not processed for user `f718a9c5980641d3b67ce061de573c71`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:12.671014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:12.803913 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2254/4455] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:12.813654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:12.814052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:12.814480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:12.837700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] MFA Rules not processed for user `f718a9c5980641d3b67ce061de573c71`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:12.838944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:12.850156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2202/4456] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:12.946107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:12.948553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:13.025103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:13.026968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:13.048797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e42c2-48b0-4df2-acd5-f51b0bb6665c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:13.056587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:13.057218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:13.057505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:13.057942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:13.067702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:13.070929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:13.077405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:13.085843 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2255/4457] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:13.164370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4f47b-39e5-45c4-a962-1bf20c1746f2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:13.172909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:13.174032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:13.174917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:13.175195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:13.178020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:13.260240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:13.261700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:13.274294 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2203/4458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:23.269415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:24.385884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:24.391726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:24.514875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423da3fd-9d94-433c-a455-ef0be9a0d832 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:24.529288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:24.529288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:24.529288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:24.529288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] PATH_INFO: `/v3/users/d142c33eccd048bfa3415b1661ee373f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:24.529702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d142c33eccd048bfa3415b1661ee373f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:24.535319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:24.623781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd142c33eccd048bfa3415b1661ee373f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:24.625248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd142c33eccd048bfa3415b1661ee373f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:24.643457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd142c33eccd048bfa3415b1661ee373f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:24.664115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd142c33eccd048bfa3415b1661ee373f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:24.664893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cba63fb-00d2-4e1b-ac40-e5bf4156ce21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd142c33eccd048bfa3415b1661ee373f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:24.679644 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2256/4459] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:07:24 2022] DELETE /identity/v3/users/d142c33eccd048bfa3415b1661ee373f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:25.092749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:25.109645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdad983-3599-4baa-99c6-5f9c350ed178 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:25.255356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:25.257845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.258341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.258833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] PATH_INFO: `/v3/projects/b0c1b7c4c6ff480c82a1f9708b7dbe62` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.263387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0c1b7c4c6ff480c82a1f9708b7dbe62)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:25.268612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:25.356677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0c1b7c4c6ff480c82a1f9708b7dbe62', 'request_id': 'req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:25.366279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0c1b7c4c6ff480c82a1f9708b7dbe62', 'request_id': 'req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:25.376034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0c1b7c4c6ff480c82a1f9708b7dbe62', 'request_id': 'req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:25.399228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] The token cache is being invalidate because project b0c1b7c4c6ff480c82a1f9708b7dbe62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:07:25.399894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:25.408251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2f6b488b-93d8-4ff5-94bb-cdcd7a387789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:25.417517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2204/4460] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:07:25 2022] DELETE /identity/v3/projects/b0c1b7c4c6ff480c82a1f9708b7dbe62 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:25.419872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.420291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.420434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.456428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:25.457683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:25.555210 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2257/4461] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:25.558315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.560188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.560491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.602198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:25.603683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:25.618621 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2205/4462] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:25.618833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:25.652358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bdf7-2054-4043-9c51-af7769b5dc6f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:25.659075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:25.659778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.660061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.660765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.661447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:07:25.661757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:25.664540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:25.693976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2258/4463] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:07:25.697196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:25.713970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49cfa4-d6fd-4e38-97cc-43d90fa82de6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:25.726559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:25.728274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.728597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.728865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.729715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:25.732140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:25.776551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2206/4464] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:07:25 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:07:25.781559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:25.798340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512c352-a6e6-4c47-ba26-e2fa9e777a26 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:25.808957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:25.921461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:25.922375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.923143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.923594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.924459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:25.927807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:25.961501 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2259/4465] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:07:25 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:07:25.962918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:25.978025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410d9ac-ec54-4428-83d8-89829b04f7da admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:25.983988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:25.984611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:25.984888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:25.985336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:25.986074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:25.987940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:26.002650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2207/4466] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:07:26.004907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:26.025016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f44eb-9a87-418c-8d8d-59912e5eac92 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:26.030604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:26.031630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:26.033317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:26.033837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:26.034437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:26.036676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15edc65-6bac-40c4-ad82-ec4aba657122 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:26.052855 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2260/4467] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:07:26.057243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:26.084954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8893e9-89cf-4b68-842e-23cb96bbb95a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:26.090758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:26.094351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:26.094351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:26.094351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] PATH_INFO: `/v3/projects/8428ee98cbdc457192d9889bf10d1036/users/fdbd9a2294414c0e932ef46d11f8e4b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:26.112900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8428ee98cbdc457192d9889bf10d1036, user_id=fdbd9a2294414c0e932ef46d11f8e4b6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:26.115609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:26.133321 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2208/4468] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:07:26 2022] PUT /identity/v3/projects/8428ee98cbdc457192d9889bf10d1036/users/fdbd9a2294414c0e932ef46d11f8e4b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:26.138370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e8f83-4a59-4ed6-b30b-f3a61b09c3fb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:26.138689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e8f83-4a59-4ed6-b30b-f3a61b09c3fb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:26.139032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e8f83-4a59-4ed6-b30b-f3a61b09c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:26.161232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a58e8f83-4a59-4ed6-b30b-f3a61b09c3fb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:26.162460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a58e8f83-4a59-4ed6-b30b-f3a61b09c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:26.245930 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2261/4469] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:26.265474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:26.269448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:26.307970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:26.347889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9aee3f-ccd4-4597-9ae2-9c206d7a3784 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:26.353124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:26.354143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:26.354664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:26.354988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:26.408867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:26.411047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1e2dd-f3b6-4de9-8080-56cd78bb23b8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:26.425536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2209/4470] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:28.792465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:28.792987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:28.793217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:28.829651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] MFA Rules not processed for user `370b057b12c148cdbb673ba6c06bd6b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:28.834029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:29.001750 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2262/4471] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:07:29.338425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854df5b-3e09-4a6d-bf11-56e2f3ca66da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:29.338925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854df5b-3e09-4a6d-bf11-56e2f3ca66da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:29.339130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854df5b-3e09-4a6d-bf11-56e2f3ca66da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:29.383242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6854df5b-3e09-4a6d-bf11-56e2f3ca66da None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:29.384612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6854df5b-3e09-4a6d-bf11-56e2f3ca66da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:29.486449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2210/4472] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:29.548424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:29.549872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:29.601715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:29.677695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6b3b1-72d2-40a5-b968-e840ab7e9d0e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:29.684031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:29.684654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:29.684932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:29.685277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:29.694959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:29.696556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016f6e7-0742-4cb7-8912-d084795fb32e service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:29.712184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2263/4473] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:07:38.573076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:38.573580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:38.574032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:38.650183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] MFA Rules not processed for user `fdbd9a2294414c0e932ef46d11f8e4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:38.651441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:38.841402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2211/4474] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:38.851300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:38.852294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:38.852783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:38.888758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] MFA Rules not processed for user `fdbd9a2294414c0e932ef46d11f8e4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:38.890355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:38.900957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2264/4475] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:38.913266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:38.916625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:39.078040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:39.098285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbfff1-4cae-48fb-b848-b938a0956f6a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:39.106382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:39.107038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:39.107470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:39.107739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:39.129196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:39.130853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:39.146933 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2212/4476] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:42.891794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:42.893604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:43.170330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ddca-242e-421b-81bd-5d612aaac8e5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:43.193437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:43.194266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:43.194699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:43.195067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:43.196982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:43.219529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:43.221187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:43.237680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2265/4477] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:49.137757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:53.229081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=62 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:54.147453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:54.163806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921efdc2-6a84-4042-8cab-645268062605 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:54.169966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:54.170636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:54.170913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:54.171432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:54.174309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:54.192698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:54.195086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941e178-f34f-4d8d-aca7-5c6e1681affe service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:54.228872 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2213/4478] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:58.015845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:58.017540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.040623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c72bab-02a6-49c4-9e41-cbfd2164e405 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:58.057886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:58.058475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:58.058792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:58.059072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] PATH_INFO: `/v3/users/cddcfb5cd4c74808b09a1ca4358fc4c1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:58.060328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cddcfb5cd4c74808b09a1ca4358fc4c1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:58.063029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:58.178879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cddcfb5cd4c74808b09a1ca4358fc4c1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.179560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cddcfb5cd4c74808b09a1ca4358fc4c1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.189986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cddcfb5cd4c74808b09a1ca4358fc4c1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.206753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cddcfb5cd4c74808b09a1ca4358fc4c1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.207675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cddcfb5cd4c74808b09a1ca4358fc4c1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.229411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:58.230060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2266/4479] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:07:58 2022] DELETE /identity/v3/users/cddcfb5cd4c74808b09a1ca4358fc4c1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:58.230594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:58.231215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:58.282571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:58.286302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.330664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:58.332313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.446998 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2214/4480] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:58.508241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e81b-c029-43a9-a1af-b2a2ced6a8cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:58.514811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:58.517787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:58.520428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.531536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:58.532816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:58.534323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] PATH_INFO: `/v3/users/f718a9c5980641d3b67ce061de573c71` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:58.539939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f718a9c5980641d3b67ce061de573c71)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:58.545081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:58.669722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f718a9c5980641d3b67ce061de573c71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.670307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f718a9c5980641d3b67ce061de573c71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.679993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd2620-8ff2-4d61-971c-6ae8251a58cd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:58.686539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:58.688078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:58.690313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f718a9c5980641d3b67ce061de573c71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.695216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:58.695542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:58.702204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.721658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f718a9c5980641d3b67ce061de573c71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.722407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f52a5d9b-6bbc-47a5-b92c-34919e073039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f718a9c5980641d3b67ce061de573c71'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:07:58.725954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:07:58.731934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:58.733401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:58.751719 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2267/4481] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:07:58 2022] DELETE /identity/v3/users/f718a9c5980641d3b67ce061de573c71 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:07:58.756947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:58.757239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:58.757516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:58.767187 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2215/4482] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:07:58.787478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:07:58.788936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:58.953970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2268/4483] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:07:58.986902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:07:58.988338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:07:59.015536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abd10-1296-45a3-98ff-cdd8d768f5d9 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:07:59.187983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:07:59.188611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:07:59.188944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:07:59.189146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:07:59.214186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:07:59.215705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:07:59.255174 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2216/4484] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:00.426168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:00.427826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:00.435345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:00.437468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:00.445766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7aef5-5075-46e3-80f2-a6c2790d457a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:00.457905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115225-9060-4ab3-be1f-88b25402e397 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:00.458233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:00.459348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.459642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.459918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] PATH_INFO: `/v3/projects/3fbcb4a735d84e789094154184e8d622` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.463370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fbcb4a735d84e789094154184e8d622)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:00.468953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:00.499895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:00.503604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.504528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.507465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] PATH_INFO: `/v3/projects/28e486847ec742fbbd1ed13b404eb519` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.509855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e486847ec742fbbd1ed13b404eb519)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:00.515932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:00.596161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:00.606289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fbcb4a735d84e789094154184e8d622', 'request_id': 'req-36034341-0bde-4ae0-b9e7-89245131fc76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.626976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e486847ec742fbbd1ed13b404eb519', 'request_id': 'req-1207e800-864b-4287-b6d2-1c1eb9b88254'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.636852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fbcb4a735d84e789094154184e8d622', 'request_id': 'req-36034341-0bde-4ae0-b9e7-89245131fc76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.640854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e486847ec742fbbd1ed13b404eb519', 'request_id': 'req-1207e800-864b-4287-b6d2-1c1eb9b88254'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.649713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fbcb4a735d84e789094154184e8d622', 'request_id': 'req-36034341-0bde-4ae0-b9e7-89245131fc76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.659304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e486847ec742fbbd1ed13b404eb519', 'request_id': 'req-1207e800-864b-4287-b6d2-1c1eb9b88254'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.683104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] The token cache is being invalidate because project 3fbcb4a735d84e789094154184e8d622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:08:00.683796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.690064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.694556 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2269/4485] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:08:00 2022] DELETE /identity/v3/projects/3fbcb4a735d84e789094154184e8d622 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:00.695396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] The token cache is being invalidate because project 28e486847ec742fbbd1ed13b404eb519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:08:00.695706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.696256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:00.696376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1207e800-864b-4287-b6d2-1c1eb9b88254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:00.697930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2217/4486] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:08:00 2022] DELETE /identity/v3/projects/28e486847ec742fbbd1ed13b404eb519 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:00.698422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:00.704722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e835-101f-4943-b2d5-d7ae20a773f5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.705003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e835-101f-4943-b2d5-d7ae20a773f5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.705271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e835-101f-4943-b2d5-d7ae20a773f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.759426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9c74e835-101f-4943-b2d5-d7ae20a773f5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:00.761255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9c74e835-101f-4943-b2d5-d7ae20a773f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:00.788101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034341-0bde-4ae0-b9e7-89245131fc76 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:00.809804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:00.875052 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2218/4487] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:00.876879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.877201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.877476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.901900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:00.902143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:00.902464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.902779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.903057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] PATH_INFO: `/v3/users/4653229d038e4ee2a9488c82a7ae87c8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.903645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:00.904941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4653229d038e4ee2a9488c82a7ae87c8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:00.911672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:00.913683 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2219/4488] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:00.918224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:00.934724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039dca-13a4-420d-9f5b-a0934d3e1240 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:00.941189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:00.942896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:00.943192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:00.943640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:00.944315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:08:00.944608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:00.946538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:00.971484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2220/4489] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:08:00.982025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.005057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4653229d038e4ee2a9488c82a7ae87c8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.006242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4653229d038e4ee2a9488c82a7ae87c8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.010032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28a691-d742-463b-a1f9-c74a6243b1d9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.021646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.022422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.022892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.023441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.024533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.026147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4653229d038e4ee2a9488c82a7ae87c8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.029631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.050791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4653229d038e4ee2a9488c82a7ae87c8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.052601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4653229d038e4ee2a9488c82a7ae87c8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.075380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2270/4490] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:08:00 2022] DELETE /identity/v3/users/4653229d038e4ee2a9488c82a7ae87c8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:08:01.086144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2221/4491] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:08:00 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:01.090677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.092762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.200984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b091ce-57c8-41c8-b552-8f905910c252 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.312281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:01.330450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.331274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.331613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.331913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.332715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.335458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.390678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.401968 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2271/4492] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:08:01 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:08:01.432835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad82840-1fbf-4756-bdc9-7be13e36281d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.456483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.458775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.460033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.460737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.461489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.466538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.472819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.475313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.509449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8ec0-0c0a-473b-b511-e51e747a0477 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.539479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.540293 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2222/4493] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:01.541634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.542171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.542451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] PATH_INFO: `/v3/projects/129c4ea8234040c5883fd66abcd5fcbe` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.543940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.552527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129c4ea8234040c5883fd66abcd5fcbe)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.554451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.569504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2ac24-7bcf-4446-ba37-e79c34202704 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.576842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.578053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.578328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.578809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.581189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.583409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.600741 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2223/4494] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:01.604961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.622972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929077c8-afa7-4bf2-82e7-3e6a315f6039 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.665069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129c4ea8234040c5883fd66abcd5fcbe', 'request_id': 'req-011abd2e-9c2b-4a09-9973-2940f7cc6b16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.684637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129c4ea8234040c5883fd66abcd5fcbe', 'request_id': 'req-011abd2e-9c2b-4a09-9973-2940f7cc6b16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.694260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:01.696134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129c4ea8234040c5883fd66abcd5fcbe', 'request_id': 'req-011abd2e-9c2b-4a09-9973-2940f7cc6b16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.712284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.713064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.713351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.713616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] PATH_INFO: `/v3/projects/b170a25f4e4d44ba88fffcdc51d3fbbe/users/a22830d73cb043b8b9c2e070afa7af55/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.715623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] The token cache is being invalidate because project 129c4ea8234040c5883fd66abcd5fcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:08:01.716047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.716719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-011abd2e-9c2b-4a09-9973-2940f7cc6b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:01.719346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2272/4495] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:08:01 2022] DELETE /identity/v3/projects/129c4ea8234040c5883fd66abcd5fcbe => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:01.723593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058935-90ca-4a2b-9a03-8eeead7069bd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.723951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058935-90ca-4a2b-9a03-8eeead7069bd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.724301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae058935-90ca-4a2b-9a03-8eeead7069bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.735450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b170a25f4e4d44ba88fffcdc51d3fbbe, user_id=a22830d73cb043b8b9c2e070afa7af55, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.739369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8505b59-273f-4e61-ae0d-46d8cb9baa0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:01.750678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ae058935-90ca-4a2b-9a03-8eeead7069bd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:01.752056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae058935-90ca-4a2b-9a03-8eeead7069bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.757326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2224/4496] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:08:01 2022] PUT /identity/v3/projects/b170a25f4e4d44ba88fffcdc51d3fbbe/users/a22830d73cb043b8b9c2e070afa7af55/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:08:01.762895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.763145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.763573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.789799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:01.789799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.849780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2273/4497] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:01.859544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.860064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.860526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.907503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:01.911428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.929966 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2225/4498] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:08:01.936062 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2274/4499] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:08:01.942133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.942952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:01.944368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:01.962304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33c53a-0d7f-4084-a044-b7504f7adf15 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:01.982329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:01.986297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:01.986960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:01.987804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:01.992112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:08:01.992456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:01.994390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.019382 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2226/4500] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:08:02.024171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.058525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc77a02-07da-4c0f-b83a-59a791d3f110 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.067522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff3937-0d2c-4211-97e0-7c471d052428 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.069482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.070539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.071259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.071689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.072384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.072692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.073195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.073573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.073944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.077042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.088841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.090336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.095809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:02.113753 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2275/4501] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:08:02.119152 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2227/4502] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:08:02 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:02.121720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.135517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e648dd-148b-45c7-966b-ca959323d2e5 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.214405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.215227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.215957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.215957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.216593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.219644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.247729 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2276/4503] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:08:02 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:02.250294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.261751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20bf62-bb0a-42ba-afdc-2d710d0964f9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.267047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.267677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.267958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.268235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.268884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.270581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.288554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2228/4504] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:02.292182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.306101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f08a3-6b10-443e-bcd8-714ab91f1e0c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.312727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.314021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.314321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.314712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.315462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.318049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b27d11-b93d-4a13-9b56-06b9ff24e3c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.330184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2277/4505] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:08:02.333697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.346549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d008e27-3a82-4681-9df1-2c7483a20804 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.351503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.352184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.352464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.352739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] PATH_INFO: `/v3/projects/fa182bfe232e42bc8c4b890b3072fabb/users/cf5f3d705dff4d7eadbd8b2b6082b605/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.371447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa182bfe232e42bc8c4b890b3072fabb, user_id=cf5f3d705dff4d7eadbd8b2b6082b605, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.374946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.391838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2229/4506] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:08:02 2022] PUT /identity/v3/projects/fa182bfe232e42bc8c4b890b3072fabb/users/cf5f3d705dff4d7eadbd8b2b6082b605/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:02.402078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ba9dd-3c65-4fad-883c-e912af198816 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.402379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ba9dd-3c65-4fad-883c-e912af198816 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.402685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ba9dd-3c65-4fad-883c-e912af198816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.487645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-967ba9dd-3c65-4fad-883c-e912af198816 None None] MFA Rules not processed for user `cf5f3d705dff4d7eadbd8b2b6082b605`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:02.490157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-967ba9dd-3c65-4fad-883c-e912af198816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:02.697197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2278/4507] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:02.708101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:02.854587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d169ce-1828-46b8-bd87-b0334b245acd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:02.858999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:02.875617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:02.876080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:02.876429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:02.886035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:02.887637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96e32-7283-4350-bc5f-ce7ea8ecb134 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:02.899606 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2230/4508] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:08.412650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d395-2db1-4701-bf0a-56804e038d5e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:08.413176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d395-2db1-4701-bf0a-56804e038d5e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:08.413176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d395-2db1-4701-bf0a-56804e038d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:08.442681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b935d395-2db1-4701-bf0a-56804e038d5e None None] MFA Rules not processed for user `9eca15a31e064f4199bfc4b04de7c5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:08.443916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b935d395-2db1-4701-bf0a-56804e038d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:08.528149 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2279/4509] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:12.895030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:15.346842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:15.347432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:15.347432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:15.407779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:15.422872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:15.595306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2231/4510] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:15.603405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:15.603849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:15.603849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:15.666523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:15.669398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:15.754356 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2280/4511] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:15.760868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:15.762460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:15.944366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ce3a4-17e0-4fd6-86e9-5cdec6c51c63 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:15.973767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:15.974874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:15.975853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:15.976342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:15.987552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:15.989674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:16.003689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2232/4512] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:17.391143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:17.395947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:17.411846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:17.426047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ed60b-c741-41bb-96be-3cad84a7d6ca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:17.435373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:17.436121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:17.436505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:17.436788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:17.438628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:17.450858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:17.455206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfc2b5-2c7a-4ea8-a76d-680d76d664e2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:17.475652 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2281/4513] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:24.272459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:24.371585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb13542-c506-4911-843f-c312d78b0e69 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:24.377233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:24.377875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:24.378156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:24.378432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] PATH_INFO: `/v3/users/cf5f3d705dff4d7eadbd8b2b6082b605` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:24.379611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5f3d705dff4d7eadbd8b2b6082b605)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:24.381605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:24.402889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:24.506040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5f3d705dff4d7eadbd8b2b6082b605'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:24.508257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5f3d705dff4d7eadbd8b2b6082b605'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:24.524571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5f3d705dff4d7eadbd8b2b6082b605'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:24.537260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5f3d705dff4d7eadbd8b2b6082b605'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:24.538167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5f3d705dff4d7eadbd8b2b6082b605'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:24.568344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:24.568827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:24.569010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:24.573336 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2233/4514] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:08:24 2022] DELETE /identity/v3/users/cf5f3d705dff4d7eadbd8b2b6082b605 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:24.609852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:24.611140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:24.720155 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2282/4515] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:24.736700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:24.738338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:24.829097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005676e-d352-4e7b-b3bf-a8bac3ebdd9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:24.834291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:24.835552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:24.835984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:24.836357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:24.855318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:24.856787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5860e-aac9-484c-a279-47034b01a5e0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:24.867643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2234/4516] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:25.109542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.110886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:25.132502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b7c5d-67ee-4de6-a245-463be4ac7571 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.138424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.147206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.147545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.147865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] PATH_INFO: `/v3/projects/fa182bfe232e42bc8c4b890b3072fabb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.149147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa182bfe232e42bc8c4b890b3072fabb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.151721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.212766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa182bfe232e42bc8c4b890b3072fabb', 'request_id': 'req-e87daae4-b567-401a-ba90-2dfc5623e3cd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:25.221385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa182bfe232e42bc8c4b890b3072fabb', 'request_id': 'req-e87daae4-b567-401a-ba90-2dfc5623e3cd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:25.228467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa182bfe232e42bc8c4b890b3072fabb', 'request_id': 'req-e87daae4-b567-401a-ba90-2dfc5623e3cd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:25.244351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] The token cache is being invalidate because project fa182bfe232e42bc8c4b890b3072fabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:08:25.244687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:25.245659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e87daae4-b567-401a-ba90-2dfc5623e3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:25.253573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.253833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.254068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.257305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2283/4517] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:08:25 2022] DELETE /identity/v3/projects/fa182bfe232e42bc8c4b890b3072fabb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:25.281730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:25.283712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:25.437873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2235/4518] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:25.447396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.447635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.448444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.485371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:25.487016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:25.498540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2284/4519] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:25.502700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.513500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5102d1-88f7-4c6c-9c38-6b25954de844 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.520208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.521122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.521403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.521674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.522367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:08:25.522805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.525203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.550410 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2236/4520] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:08:25.551111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.562589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76bcd8-30ee-42b8-ba64-68f805fbac21 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.567529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.568145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.568427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.568964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.569320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.574016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.622717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2285/4521] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:08:25 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:25.627696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.670362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6f59-deee-4ce0-ad8f-6b07bc6d530d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.715799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:25.804448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.804448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.804839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.808562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.809286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.814556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.864672 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2237/4522] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:08:25 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:25.865458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.881449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2059a1-9af4-4541-8faf-aa15c50a64f1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.887513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.888328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.889334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.889786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.890936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.892924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.908050 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2286/4523] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:25.912211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.923007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7da3c-2d62-4166-9973-46f444aa6bd4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.927805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.928401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.928677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.929056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.929687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.931321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032d010-73cc-49e8-b545-961beada753e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:25.942520 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2238/4524] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:25.946873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:25.959180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567668c-0b4e-43e7-ad4c-b1ff456ee15f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:25.964665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:25.965710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:25.966314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:25.966796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] PATH_INFO: `/v3/projects/f6023321fef14d9a9b05fbfa68304359/users/9d35200883dc4b75afd1937ac6dad4e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:25.987979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6023321fef14d9a9b05fbfa68304359, user_id=9d35200883dc4b75afd1937ac6dad4e4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:25.991653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:26.010612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2287/4525] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:08:25 2022] PUT /identity/v3/projects/f6023321fef14d9a9b05fbfa68304359/users/9d35200883dc4b75afd1937ac6dad4e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:26.021660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:26.021948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:26.022205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:26.058056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] MFA Rules not processed for user `9d35200883dc4b75afd1937ac6dad4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:26.059326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:26.146617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2239/4526] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:26.165996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:26.167959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:26.184492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:26.268096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e8c52-120b-422f-a748-00faf2601650 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:26.274652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:26.275372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:26.275719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:26.276168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:26.292961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:26.294636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8538116-4a93-4f74-b483-07f386d1b971 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:26.306309 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2288/4527] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:27.839493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:27.840932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:28.081836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:28.087497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ef93e-0fb1-487c-bf3b-de8544898334 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:28.119675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:28.120309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:28.120593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:28.120866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:28.122754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:28.210829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:28.212416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:28.233861 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2240/4528] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:32.491646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3457e2-e168-47b5-80fb-ef656a16b599 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:32.492753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3457e2-e168-47b5-80fb-ef656a16b599 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:32.493070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3457e2-e168-47b5-80fb-ef656a16b599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:32.531839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0a3457e2-e168-47b5-80fb-ef656a16b599 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:32.533342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a3457e2-e168-47b5-80fb-ef656a16b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:32.658671 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2289/4529] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:32.750625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:32.752237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:32.877653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b67f34-e959-4ab3-8140-d2cdb237c117 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:32.892094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:32.892682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:32.893625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:32.894082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:32.906617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:32.910305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:32.932416 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2241/4530] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:08:33.011393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e32ee-17ee-4b72-9042-773ace6f06fd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:33.011677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e32ee-17ee-4b72-9042-773ace6f06fd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:33.011967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e32ee-17ee-4b72-9042-773ace6f06fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:33.060710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-655e32ee-17ee-4b72-9042-773ace6f06fd None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:33.062788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-655e32ee-17ee-4b72-9042-773ace6f06fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:33.204318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2290/4531] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:33.545886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:33.547471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:33.727222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4a70-92c1-4401-82a1-ea5b38c4e913 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:33.735996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:33.737012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:33.737806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:33.738516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] PATH_INFO: `/v3/users/0fac2afeb1aa43b980be965a1f23f12a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:33.739952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fac2afeb1aa43b980be965a1f23f12a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:33.743801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:33.767315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:33.828246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fac2afeb1aa43b980be965a1f23f12a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:33.828979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fac2afeb1aa43b980be965a1f23f12a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:33.848858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fac2afeb1aa43b980be965a1f23f12a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:33.858327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fac2afeb1aa43b980be965a1f23f12a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:33.859343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fac2afeb1aa43b980be965a1f23f12a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:33.882316 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2242/4532] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:08:33 2022] DELETE /identity/v3/users/0fac2afeb1aa43b980be965a1f23f12a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:33.886956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:33.888297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:33.888767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:33.916606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:33.922121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:34.061200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2291/4533] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:34.071434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:34.072969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:34.181214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbcd13-5582-4ef3-ad75-33cf717d32dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:34.188908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:34.189541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:34.189823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:34.190392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:34.236867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:34.238565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010e4c-727c-41d3-a961-e49bc6187e04 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:34.261241 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2243/4534] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:34.641550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:34.644051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:34.681029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:34.689367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200cc4f6-f457-4ee0-9c26-e12a2c1d377b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:34.703960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:34.704684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:34.706682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:34.706965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] PATH_INFO: `/v3/projects/de1b6b1ac46840199aa78debc44fa00d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:34.710922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1b6b1ac46840199aa78debc44fa00d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:34.713468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:34.846058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1b6b1ac46840199aa78debc44fa00d', 'request_id': 'req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:34.853659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1b6b1ac46840199aa78debc44fa00d', 'request_id': 'req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:34.863997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1b6b1ac46840199aa78debc44fa00d', 'request_id': 'req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:34.921787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] The token cache is being invalidate because project de1b6b1ac46840199aa78debc44fa00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:08:34.923128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:34.924191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:08:34.933824 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2292/4535] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:08:34 2022] DELETE /identity/v3/projects/de1b6b1ac46840199aa78debc44fa00d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:34.934863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db3d06-2415-4fe7-9a61-cc8547b25a06 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:34.935417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db3d06-2415-4fe7-9a61-cc8547b25a06 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:34.935666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db3d06-2415-4fe7-9a61-cc8547b25a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:34.978803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:34.980539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:35.044881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-34db3d06-2415-4fe7-9a61-cc8547b25a06 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:35.046838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-34db3d06-2415-4fe7-9a61-cc8547b25a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:35.186010 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2244/4536] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:35.187637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.188253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.188600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.231787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] MFA Rules not processed for user `fdbd9a2294414c0e932ef46d11f8e4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:35.233926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:35.264145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3307de-76e2-4f1d-9d5a-0d9ef176b1f3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:35.267983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:35.279479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:35.280929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.282229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.282901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.285405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:35.301672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:35.303868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1a564-1efe-4905-a385-ca4aa13b9b67 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:35.317993 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2293/4537] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:08:35.324176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.324894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.325370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.372123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:35.372123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:35.383229 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2294/4538] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:35.384823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:35.405430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a827-f687-4004-9a08-a0243fb21465 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:35.411863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:35.412546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.412894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.413237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.413958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:08:35.414331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:35.416160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:35.421733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2245/4539] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:35.459374 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2295/4540] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:08:35.461710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:35.493486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3afbb-2aad-4099-beba-02ff0f16b83d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:35.516320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:35.518168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.518451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.518752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.519534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:35.525149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:35.613520 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2246/4541] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:08:35 2022] POST /identity/v3/projects => generated 359 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:35.618118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:35.648990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89488d-ebde-4e83-a138-c7eddc56f461 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:35.825115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:35.842699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:35.843412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.843693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.843966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.844606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:35.846602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:35.908504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2296/4542] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:08:35 2022] POST /identity/v3/users => generated 280 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:08:35.911232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:35.929896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad2c76-7ab8-4db6-bc1c-5687bd7ef5cc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:35.947049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:35.948575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:35.949053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:35.949511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:35.950591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:35.952817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:35.967206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2247/4543] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:35.972094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:35.986734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2efa0d-95eb-4dc9-b074-ec10595168c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:36.001418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:36.002542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:36.003030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:36.003792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:36.004684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:36.006913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a629eb6-1c72-4c84-85ca-0f86c27d1b73 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:36.029575 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2297/4544] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:08:36.029949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:36.064755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f976975f-7e07-43a7-8f9b-a528dd03cf14 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:36.093334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:36.100512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:36.102197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:36.104307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] PATH_INFO: `/v3/projects/0f0bd29c38f24ebfbf4a66462adac801/users/a79ff8f5c7554b60b606f779b7556f18/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:36.179154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0bd29c38f24ebfbf4a66462adac801, user_id=a79ff8f5c7554b60b606f779b7556f18, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:36.188638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:36.244191 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2248/4545] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:08:36 2022] PUT /identity/v3/projects/0f0bd29c38f24ebfbf4a66462adac801/users/a79ff8f5c7554b60b606f779b7556f18/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:08:36.259229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6990-891c-4912-bf13-d77021f490b2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:36.259764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6990-891c-4912-bf13-d77021f490b2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:36.260451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6990-891c-4912-bf13-d77021f490b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:36.308529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d1fb6990-891c-4912-bf13-d77021f490b2 None None] MFA Rules not processed for user `a79ff8f5c7554b60b606f779b7556f18`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:08:36.310164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1fb6990-891c-4912-bf13-d77021f490b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:36.423409 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2298/4546] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:08:36.457106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:08:36.459464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:08:36.602721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26593aa-a316-478f-be65-8daf3bddcb7c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:08:36.617913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:08:36.619539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:08:36.619539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:08:36.619539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:08:36.649969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:08:36.651811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:08:36.664246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2249/4547] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:08:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:08:46.420241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=40,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:08:46.659064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:00.566967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529929b-b95a-4141-a759-5777b026e6f6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:00.568071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529929b-b95a-4141-a759-5777b026e6f6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:00.569010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529929b-b95a-4141-a759-5777b026e6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:00.628124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7529929b-b95a-4141-a759-5777b026e6f6 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:00.631934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7529929b-b95a-4141-a759-5777b026e6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:00.735194 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2299/4548] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:01.162002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:01.164998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:01.179247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31241-d65c-4d52-8f84-5592370c716e service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:01.183950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:01.184942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:01.185401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:01.185822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:01.189806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:01.302373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:01.304607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92003f-dbe2-4e82-b070-f873e092fa16 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:01.321306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2250/4549] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:06.143343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:06.143842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:06.143938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:06.164870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:06.166109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:06.177633 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2300/4550] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:06.907806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93160e-4a55-4a05-a857-d9472f460437 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:06.908382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93160e-4a55-4a05-a857-d9472f460437 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:06.908382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93160e-4a55-4a05-a857-d9472f460437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:06.940506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1f93160e-4a55-4a05-a857-d9472f460437 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:06.942636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f93160e-4a55-4a05-a857-d9472f460437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:06.953504 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2251/4551] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:07.878872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:07.881127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:07.911809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc0537-af8a-4fb2-a4c9-e5c59683a9a8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:07.924523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:07.925228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:07.925680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:07.926102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] PATH_INFO: `/v3/users/6493da825aa24f43bdf8f0835a16e3bb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:07.930160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:07.931085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:07.931508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:07.942633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6493da825aa24f43bdf8f0835a16e3bb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:07.953729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:07.974165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:07.976196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:08.007889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2252/4552] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:08.041908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6493da825aa24f43bdf8f0835a16e3bb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.042726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6493da825aa24f43bdf8f0835a16e3bb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.059765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6493da825aa24f43bdf8f0835a16e3bb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.089749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6493da825aa24f43bdf8f0835a16e3bb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.090726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e97e4b5-156b-4c70-9b61-30fdac254b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6493da825aa24f43bdf8f0835a16e3bb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.117836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2301/4553] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:07 2022] DELETE /identity/v3/users/6493da825aa24f43bdf8f0835a16e3bb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:08.616611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:08.632803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52295df2-3508-4372-bf47-fad839f60f8a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:08.712972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253484a-0cdb-418c-b3f1-e8169eae7bfc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:08.713256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253484a-0cdb-418c-b3f1-e8169eae7bfc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:08.713529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253484a-0cdb-418c-b3f1-e8169eae7bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:08.738330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7253484a-0cdb-418c-b3f1-e8169eae7bfc None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:08.739948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7253484a-0cdb-418c-b3f1-e8169eae7bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:08.753664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:08.754465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:08.754790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:08.755060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] PATH_INFO: `/v3/projects/c1e5d6f3d41e4aa49e630f8c7ac87ded` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:08.756649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e5d6f3d41e4aa49e630f8c7ac87ded)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:08.759090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:08.807903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2302/4554] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:08.820756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e5d6f3d41e4aa49e630f8c7ac87ded', 'request_id': 'req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.836404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e5d6f3d41e4aa49e630f8c7ac87ded', 'request_id': 'req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.845889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e5d6f3d41e4aa49e630f8c7ac87ded', 'request_id': 'req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.861106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] The token cache is being invalidate because project c1e5d6f3d41e4aa49e630f8c7ac87ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:08.862192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.862900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b21240-0bc3-48fc-9cc4-a3b6543f13ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:08.864701 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2253/4555] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:08 2022] DELETE /identity/v3/projects/c1e5d6f3d41e4aa49e630f8c7ac87ded => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:08.874028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:08.874321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:08.874697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:08.896904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:08.898154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:08.909798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:08.977641 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2303/4556] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:08.979405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:08.979885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:08.980004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.004276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:09.005490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:09.017615 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2254/4557] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:09.022839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.037007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175fd56-ff38-4338-b306-2af230012f2c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.042712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.044156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.044437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.044699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.045352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:09.045648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.047278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.068807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.069039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2304/4558] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:09.074957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:09.080312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f48cf-d0d6-4f64-8f64-abda6b1ff902 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.109033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.109972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.110256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.110561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.111216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.113250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.146605 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2255/4559] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:09 2022] POST /identity/v3/projects => generated 361 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:09.150587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.162332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e96a4f-7336-4a5c-b00f-269db4712477 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.269841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.270468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.270774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.271043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.271684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.274620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c9482-8d87-4d0e-956d-3cf6d521790f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.336579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2305/4560] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:09 2022] POST /identity/v3/users => generated 281 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:09.342341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.351567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.353501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.353855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.363761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12342b88-80f1-45ab-a24c-1982a3899f92 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.375516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:09.375761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.376690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:09.376814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.376981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.377531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.379275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.383599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.419316 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2256/4561] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:09.422942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.439067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f94c-8116-4e73-9726-3b7026c61b87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.458992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.460282 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2306/4562] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:09.461756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.463052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.463668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.470638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.476880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f4c733-10ea-49c0-b9a6-b5affc5adc08 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.514001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2257/4563] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:09.518371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.536088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6d1e6-86e6-4133-8660-fe1424c2a649 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:09.541236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:09.541815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.542095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.542372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a/users/7ab52927e8594b4f8ffb59e014292a49/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.570697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa2a7852b3646a28ddf1ecc9c17513a, user_id=7ab52927e8594b4f8ffb59e014292a49, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:09.582184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:09.599321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2307/4564] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:09 2022] PUT /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a/users/7ab52927e8594b4f8ffb59e014292a49/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:09.612468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ed90-82cd-4ad6-8bf6-0b3f3488bb7a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:09.614871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ed90-82cd-4ad6-8bf6-0b3f3488bb7a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:09.615159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ed90-82cd-4ad6-8bf6-0b3f3488bb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:09.682278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e549ed90-82cd-4ad6-8bf6-0b3f3488bb7a None None] MFA Rules not processed for user `7ab52927e8594b4f8ffb59e014292a49`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:09.685330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e549ed90-82cd-4ad6-8bf6-0b3f3488bb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:09.862557 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2258/4565] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:09.875146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:09.876952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:09.904791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:10.000716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7546f6-86ca-4ec9-85da-898d749857c1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.012547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.015752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.015752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.015752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.025100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.027071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.047562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2308/4566] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:10.063408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87fd8-3d16-4755-83af-bb278b95657d None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.064593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87fd8-3d16-4755-83af-bb278b95657d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.064870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb87fd8-3d16-4755-83af-bb278b95657d None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.066217 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2259/4567] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:09:10.073891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:09:10.088789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.098751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8579f1c-c156-4b8b-8553-4f943227f410 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.110615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab52927e8594b4f8ffb59e014292a49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aa2a7852b3646a28ddf1ecc9c17513a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.110842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.110946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.111192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.112431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorizing `identity:get_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.114935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.120841 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2309/4568] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:09:10 2022] GET /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 361 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:10.149948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428385-5845-409d-bf93-564762ecd8ee None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.150228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428385-5845-409d-bf93-564762ecd8ee None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.150745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428385-5845-409d-bf93-564762ecd8ee None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.152255 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2260/4569] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:09:10.158897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:09:10.177591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.187670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cb2d-063a-4fa1-9588-c00a31b1edef tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.192246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab52927e8594b4f8ffb59e014292a49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aa2a7852b3646a28ddf1ecc9c17513a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.193482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.193763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.194179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.195402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorizing `identity:get_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.197199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240a6c0-ac91-438a-9fad-ecc663e09381 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.219462 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2310/4570] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:09:10 2022] GET /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 361 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:10.234182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467d12-ec16-4ac3-a984-27652eb49cc1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.234466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467d12-ec16-4ac3-a984-27652eb49cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.234783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467d12-ec16-4ac3-a984-27652eb49cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.242111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.242395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.242895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.244887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2311/4571] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:09:10.250612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:09:10.253515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:10.263211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a467d12-ec16-4ac3-a984-27652eb49cc1 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:10.264442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a467d12-ec16-4ac3-a984-27652eb49cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:10.280325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.293552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b239d-07c6-476f-81ef-8c9834920a0b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.298978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab52927e8594b4f8ffb59e014292a49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aa2a7852b3646a28ddf1ecc9c17513a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.299902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.301141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.301621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.303640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorizing `identity:get_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.305822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b90f31-776d-4a04-91b6-9ee9c1dc7f47 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.318801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2312/4572] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:09:10 2022] GET /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 361 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:10.363086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.363365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.363643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.368940 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2313/4573] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:09:10 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:09:10.373998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:09:10.378965 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2261/4574] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:10.409409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.447310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a68c7-3ce1-478d-a77a-dfbb6d059bba None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.452429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab52927e8594b4f8ffb59e014292a49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aa2a7852b3646a28ddf1ecc9c17513a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.453066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.453458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.453772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.463403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorizing `identity:get_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.469706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.473843 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2314/4575] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:09:10 2022] GET /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 361 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:10.528204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8d38b-6879-4904-b3d1-472d30815314 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.529949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8d38b-6879-4904-b3d1-472d30815314 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.530234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8d38b-6879-4904-b3d1-472d30815314 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.532299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2262/4576] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:09:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:09:10.538949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:09:10.559840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.569669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79524d7c-94b1-4771-8241-05ec40ad3d42 tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.574824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ab52927e8594b4f8ffb59e014292a49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aa2a7852b3646a28ddf1ecc9c17513a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.575484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.575764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.576041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.577264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorizing `identity:get_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.579216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213bdde-75d5-4f88-aeba-fbb0d7d5665d tempest-QuotasTestJSON-71753034 tempest-QuotasTestJSON-71753034-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.581776 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2315/4577] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:09:10 2022] GET /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 361 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:10.631041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8d38b-6879-4904-b3d1-472d30815314 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:10.723937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:10.733608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8d38b-6879-4904-b3d1-472d30815314 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:10.739642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:10.740643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.741171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.741883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] PATH_INFO: `/v3/users/7ab52927e8594b4f8ffb59e014292a49` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.743673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab52927e8594b4f8ffb59e014292a49)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:10.746538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:10.818874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab52927e8594b4f8ffb59e014292a49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:10.832418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab52927e8594b4f8ffb59e014292a49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:10.846702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab52927e8594b4f8ffb59e014292a49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:10.855643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab52927e8594b4f8ffb59e014292a49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:10.856484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab52927e8594b4f8ffb59e014292a49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:10.880333 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2263/4578] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:10 2022] DELETE /identity/v3/users/7ab52927e8594b4f8ffb59e014292a49 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:10.884997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:10.885443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:10.886147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:10.911267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:10.912786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:10.990621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2316/4579] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:11.018031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:11.019597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:11.112640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4456-0928-42de-a888-1d81c34b134b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:11.120308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:11.121712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:11.122222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:11.122706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:11.138617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:11.142445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1e52f-a76b-4f94-a116-17686143c184 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:11.155372 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2264/4580] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:11.915495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:11.916901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:11.930297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f99a6-03f9-406c-8d50-0d679acbea66 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:11.942396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:11.943128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:11.943413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:11.943687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] PATH_INFO: `/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:11.946310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa2a7852b3646a28ddf1ecc9c17513a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:11.948381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.045692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa2a7852b3646a28ddf1ecc9c17513a', 'request_id': 'req-0362eb9e-93d8-46f4-ada5-1b62e4741745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:12.047198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:12.055987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa2a7852b3646a28ddf1ecc9c17513a', 'request_id': 'req-0362eb9e-93d8-46f4-ada5-1b62e4741745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:12.068647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa2a7852b3646a28ddf1ecc9c17513a', 'request_id': 'req-0362eb9e-93d8-46f4-ada5-1b62e4741745'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:12.109412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] The token cache is being invalidate because project 1aa2a7852b3646a28ddf1ecc9c17513a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:12.109696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:12.110680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0362eb9e-93d8-46f4-ada5-1b62e4741745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:12.114755 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2317/4581] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:11 2022] DELETE /identity/v3/projects/1aa2a7852b3646a28ddf1ecc9c17513a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:12.121010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.121596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.121596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.156291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:12.158143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:12.251918 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2265/4582] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:12.262707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.263312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.263738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.296959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:12.299388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:12.313767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2318/4583] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:12.317876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.330751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f16f-8335-4222-a914-7176cf9c1d95 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.337499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.338375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.339011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.339533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.341070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:12.341737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.343584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59768b9a-1895-426d-a9f2-776cd1a358f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.367240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2266/4584] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:12.372348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.396115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395f239-baed-4f59-bb4b-55f35a4b2cea None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.401333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.401942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.402284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.404565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.404565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.414830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.475167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.475562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.475894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.487535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:12.489738 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2319/4585] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:12.495441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.519985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:12.521046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:12.615670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8d3ad-3294-4b1c-a2da-7df89041c4c1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.627554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.646919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.647486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.647916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.648920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.651876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.655823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2267/4586] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:12.694855 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2320/4587] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:12.695160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.711764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889401d-0ec8-4ede-8324-96b6ebf766d1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.723189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.724002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.724630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.725697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.729501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.738361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.765639 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2268/4588] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:12.770031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.786325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5473158-1815-402b-a056-1f646279c3c7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.793923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.795360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.795740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.796214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.797017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.799946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7708b0-044d-463d-926b-fc7c4ae6f3cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.818251 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2321/4589] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:12.819145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:12.831020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d3b31-ca79-42fd-85fe-7390a3102b97 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:12.836299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:12.836693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.836976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.837248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] PATH_INFO: `/v3/projects/c4ff4a00aa31492e9dd9c89044491436/users/d85cf31166d04a4892e26eee76951762/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.866896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ff4a00aa31492e9dd9c89044491436, user_id=d85cf31166d04a4892e26eee76951762, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:12.869593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:12.926456 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2269/4590] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:12 2022] PUT /identity/v3/projects/c4ff4a00aa31492e9dd9c89044491436/users/d85cf31166d04a4892e26eee76951762/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:12.938107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:12.938670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:12.938670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:12.992332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] MFA Rules not processed for user `d85cf31166d04a4892e26eee76951762`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:12.993799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:13.134299 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2322/4591] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:13.317421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:13.319371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:13.462814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0d8cb-c93e-4b07-acbc-7c7e4247721e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:13.472334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:13.473235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:13.473589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:13.473937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:13.483487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:13.485192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc7f81-88ac-40d0-b4ef-fd153e068315 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:13.495319 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2270/4592] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:09:15.821860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:15.859294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:15.974288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4842e-c6b6-4444-b37c-44ff8dfc11f7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:15.987214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:15.991618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:15.991618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:15.991618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] PATH_INFO: `/v3/users/d85cf31166d04a4892e26eee76951762` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:15.991618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85cf31166d04a4892e26eee76951762)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:15.992608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:16.163509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85cf31166d04a4892e26eee76951762'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:16.164218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85cf31166d04a4892e26eee76951762'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:16.175986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85cf31166d04a4892e26eee76951762'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:16.203241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85cf31166d04a4892e26eee76951762'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:16.204809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85cf31166d04a4892e26eee76951762'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:16.245681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2323/4593] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:15 2022] DELETE /identity/v3/users/d85cf31166d04a4892e26eee76951762 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:16.247945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:16.248409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:16.249438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:16.311006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:16.313633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:16.409314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:16.421761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2271/4594] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:16.477631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:16.479388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:16.602358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0919e-1471-4435-9e6d-5d0b09dbfdb8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:16.607208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:16.608295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:16.608805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:16.609281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:16.648734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:16.651262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab2d7a-d8ca-4783-9bd9-3f32d7fc443d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:16.663633 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2324/4595] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:09:17.973344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:17.974942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:17.989017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6483b-f1f3-405f-88a9-812ee99693df None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:17.997324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:17.998202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:17.998840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:17.999248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] PATH_INFO: `/v3/projects/c4ff4a00aa31492e9dd9c89044491436` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.001455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ff4a00aa31492e9dd9c89044491436)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.005040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.075505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ff4a00aa31492e9dd9c89044491436', 'request_id': 'req-505f7a70-8d0b-4a88-b28e-846487ffdf9a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:18.086524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ff4a00aa31492e9dd9c89044491436', 'request_id': 'req-505f7a70-8d0b-4a88-b28e-846487ffdf9a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:18.115240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ff4a00aa31492e9dd9c89044491436', 'request_id': 'req-505f7a70-8d0b-4a88-b28e-846487ffdf9a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:18.141469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] The token cache is being invalidate because project c4ff4a00aa31492e9dd9c89044491436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:18.142621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:18.145441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-505f7a70-8d0b-4a88-b28e-846487ffdf9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:18.150478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2272/4596] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:17 2022] DELETE /identity/v3/projects/c4ff4a00aa31492e9dd9c89044491436 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:18.158278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.158841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.158841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.210153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:18.212245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:18.297563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:18.346446 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2325/4597] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:18.348087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.348395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.348907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.398408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:18.399637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:18.450001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2273/4598] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:18.454418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.471413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc692e9-edb4-4f00-9af8-6c45a95fe0f6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.482444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.483233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.483759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.484180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.485554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:18.485906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.491574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.514448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2326/4599] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:18.520066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.546940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15725fc0-5b39-45ee-b120-d07eef974d68 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.553505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.554309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.555403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.556155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.557445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.560765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.614906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2274/4600] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:18 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:18.619178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.636174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa820c75-f1fb-4051-9411-03f484c204e4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.769112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.769801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.770223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.770526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.772827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.775742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.821490 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2327/4601] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:18 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:18.825658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.838648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29ad1e-a5c2-4ce6-990c-043c0b908e19 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.848355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.848957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.849243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.849516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.850165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.853458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.870342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2275/4602] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:18.875091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.892155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ff236-d1f5-4574-ae71-1f06bb1c798a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.903537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.904252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.904531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.905313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:18.906103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:18.908482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacde222-8d7f-45bd-8081-8069824ebc8d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:18.930767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2328/4603] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:18.934963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:18.953852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dc962-8c4c-41e8-8e6e-7c832cc8a2bf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:18.959510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:18.960311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:18.961060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:18.961529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] PATH_INFO: `/v3/projects/dde1e0cf8746411a81e4c2d3355715f4/users/fc125842931a40f8b05f6dde304390ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.019918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde1e0cf8746411a81e4c2d3355715f4, user_id=fc125842931a40f8b05f6dde304390ff, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:19.035623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:19.063192 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2276/4604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:18 2022] PUT /identity/v3/projects/dde1e0cf8746411a81e4c2d3355715f4/users/fc125842931a40f8b05f6dde304390ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:19.076848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.077056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.077207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.126312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] MFA Rules not processed for user `fc125842931a40f8b05f6dde304390ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:19.131470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:19.341901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2329/4605] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:19.346819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:19.459409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9442e0-07b0-4aa3-92b0-5b20562fcc13 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:19.466182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:19.467501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.468247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.468737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.469673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:19.472502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:19.530567 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2277/4606] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:19 2022] POST /identity/v3/projects => generated 401 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:19.531332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:19.549813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07632d22-d936-4fc8-8940-bffd4c3916e4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:19.618753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:19.677271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:19.678147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.678550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.679116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.679801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:19.682289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93329c-affd-4084-ac74-d7b1ab82891f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:19.722235 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2330/4607] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:19 2022] POST /identity/v3/users => generated 307 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:19.726923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:19.735053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.736022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.737312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.739848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:19.748754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0c2b-1de6-4cfb-9a3b-542dab3e2737 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:19.773280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:19.774338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.774871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.775369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.776549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:19.783127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:19.789640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:19.799737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:19.818248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2278/4608] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:19.821623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:19.855804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a10c-4dc1-4fba-aacc-46465c67331b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:19.870578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:19.871612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:19.871892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:19.873322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] PATH_INFO: `/v3/projects/ee617bd6fe224272a2d8033d3081f462/users/90c26cd616814758a19838999ff22337/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:19.901062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee617bd6fe224272a2d8033d3081f462, user_id=90c26cd616814758a19838999ff22337, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:19.904597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:19.918744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2331/4609] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:19.928402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2279/4610] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:19 2022] PUT /identity/v3/projects/ee617bd6fe224272a2d8033d3081f462/users/90c26cd616814758a19838999ff22337/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:19.932436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:19.945119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8374262-aa07-499e-8952-f573bcdd3850 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:20.041580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:20.042214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.042695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.044836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.085257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:20.093808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e363e7a-5d16-4433-a965-5196c758677b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:20.131899 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2332/4611] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:20.133225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:20.150341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf380-4cf4-4e1d-b028-030b2c651b54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:20.163960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:20.164643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.164922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.165194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] PATH_INFO: `/v3/projects/ee617bd6fe224272a2d8033d3081f462/users/90c26cd616814758a19838999ff22337/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.167172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee617bd6fe224272a2d8033d3081f462, user_id=90c26cd616814758a19838999ff22337, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:20.170545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:20.197505 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2280/4612] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:20 2022] PUT /identity/v3/projects/ee617bd6fe224272a2d8033d3081f462/users/90c26cd616814758a19838999ff22337/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:20.210012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.210613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.211090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.261665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] MFA Rules not processed for user `90c26cd616814758a19838999ff22337`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:20.267850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:20.433499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:20.468049 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2333/4613] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:20.472613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:20.484574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273c1f0-e9fc-48ae-bf0f-f732a8f9057c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:20.490185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c26cd616814758a19838999ff22337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee617bd6fe224272a2d8033d3081f462', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:20.491039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.491384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.491863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.493417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:20.494828 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:20.494828 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:09:20.494828 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:09:20.497139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:20.546566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2281/4614] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:09:20 2022] POST /identity/v3/roles => generated 239 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:20.550985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:20.587312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccc5ea-4563-48cd-9206-c588029e4860 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:20.601424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c26cd616814758a19838999ff22337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee617bd6fe224272a2d8033d3081f462', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:20.602542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.603254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.603634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] PATH_INFO: `/v3/projects/dde1e0cf8746411a81e4c2d3355715f4/users/fc125842931a40f8b05f6dde304390ff/roles/513409a658584075a6dad8966863f1b5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.610151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorizing `identity:create_grant(project_id=dde1e0cf8746411a81e4c2d3355715f4, user_id=fc125842931a40f8b05f6dde304390ff, role_id=513409a658584075a6dad8966863f1b5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:20.615544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbb2cd-fc12-4758-bb91-c70c6f729459 tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:20.676301 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2334/4615] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:20 2022] PUT /identity/v3/projects/dde1e0cf8746411a81e4c2d3355715f4/users/fc125842931a40f8b05f6dde304390ff/roles/513409a658584075a6dad8966863f1b5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:20.680809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:20.825520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93339e-63bf-468a-b753-5efb4a52419a tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:20.946964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc125842931a40f8b05f6dde304390ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde1e0cf8746411a81e4c2d3355715f4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'tempest-test_role-980967470'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:20.948024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:20.948826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:20.949722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] PATH_INFO: `/v3/users/fc125842931a40f8b05f6dde304390ff/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:20.950642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: Authorizing `identity:create_application_credential(user_id=fc125842931a40f8b05f6dde304390ff)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:20.951997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca59cce-96c9-439f-9a58-b318eac7ca95 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:21.085662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:21.086540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b69e8-c274-4390-9727-8b77e0c6b745 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.086950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b69e8-c274-4390-9727-8b77e0c6b745 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.091731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b69e8-c274-4390-9727-8b77e0c6b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.141048 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2282/4616] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:09:20 2022] POST /identity/v3/users/fc125842931a40f8b05f6dde304390ff/application_credentials => generated 659 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:21.141600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dd5b69e8-c274-4390-9727-8b77e0c6b745 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:21.141600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dd5b69e8-c274-4390-9727-8b77e0c6b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:21.143566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.144039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.144871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.188259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] MFA Rules not processed for user `fc125842931a40f8b05f6dde304390ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:21.189770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:21.284158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2335/4617] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:21.354056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.354337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.354806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.379737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] MFA Rules not processed for user `370b057b12c148cdbb673ba6c06bd6b4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:21.381380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:21.481660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2283/4618] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3396 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 24 21:09:21.486336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:21.490563 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2336/4619] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:21.508920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d08d1-a374-4132-8910-f721b239057e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:21.547899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc125842931a40f8b05f6dde304390ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde1e0cf8746411a81e4c2d3355715f4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'tempest-test_role-980967470'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:21.548583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.548868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.549145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] PATH_INFO: `/v3/users/fc125842931a40f8b05f6dde304390ff/application_credentials/e4602215a6804e24a498518c8cc1ec4c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.553422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: Authorizing `identity:delete_application_credential(user_id=fc125842931a40f8b05f6dde304390ff, application_credential_id=e4602215a6804e24a498518c8cc1ec4c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:21.555004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:21.613376 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2284/4620] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:09:21 2022] DELETE /identity/v3/users/fc125842931a40f8b05f6dde304390ff/application_credentials/e4602215a6804e24a498518c8cc1ec4c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:21.617845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:21.662153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:21.664044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:21.786120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683985f-4fce-40fd-8a6c-e9cbb1975a79 tempest-ApplicationCredentialsV3AdminTest-1337601243 tempest-ApplicationCredentialsV3AdminTest-1337601243-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:21.802091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:21.803220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.803568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.803861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] PATH_INFO: `/v3/users/9d35200883dc4b75afd1937ac6dad4e4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.805121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f083844-8660-4546-aa89-6a83a4aea7a4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:21.805244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d35200883dc4b75afd1937ac6dad4e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:21.807761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:21.814326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:21.817092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c26cd616814758a19838999ff22337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee617bd6fe224272a2d8033d3081f462', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:21.817738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.818021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.818362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] PATH_INFO: `/v3/roles/513409a658584075a6dad8966863f1b5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:21.822833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorizing `identity:delete_role(role_id=513409a658584075a6dad8966863f1b5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:21.824826 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:21.824826 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:09:21.824826 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:09:21.825990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:21.890615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:21.895785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d35200883dc4b75afd1937ac6dad4e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.896721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d35200883dc4b75afd1937ac6dad4e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.910551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d35200883dc4b75afd1937ac6dad4e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.920148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Invalidating the token cache because role 513409a658584075a6dad8966863f1b5 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:21.925152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.926605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.933825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2337/4621] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:21 2022] DELETE /identity/v3/roles/513409a658584075a6dad8966863f1b5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:21.935654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:21.935905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d35200883dc4b75afd1937ac6dad4e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.937205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:21.937691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdff7ed8-1187-48f4-9901-b244b3ccee5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d35200883dc4b75afd1937ac6dad4e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:21.990582 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2285/4622] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:21 2022] DELETE /identity/v3/users/9d35200883dc4b75afd1937ac6dad4e4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:21.994962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:21.995358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:21.995832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:22.033393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:22.037204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:22.124483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81082ba-547e-4944-a482-89f746b4c49d tempest-ApplicationCredentialsV3AdminTest-325246942 tempest-ApplicationCredentialsV3AdminTest-325246942-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:22.171483 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2286/4623] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:22.180289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:22.184106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:22.290618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:22.291535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:22.291919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:22.292295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] PATH_INFO: `/v3/users/fc125842931a40f8b05f6dde304390ff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:22.296382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc125842931a40f8b05f6dde304390ff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:22.298762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:22.308739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4919-4ce0-4261-8649-658d9978b751 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:22.321931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:22.324876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:22.325608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:22.325907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:22.352203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:22.355864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1840ff-e5ca-40bd-89b2-ddb4ac0fa745 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:22.370339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc125842931a40f8b05f6dde304390ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:22.371763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc125842931a40f8b05f6dde304390ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:22.379620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2287/4624] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:22.383583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc125842931a40f8b05f6dde304390ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:22.431046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc125842931a40f8b05f6dde304390ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:22.432317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc125842931a40f8b05f6dde304390ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:22.451169 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2338/4625] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:21 2022] DELETE /identity/v3/users/fc125842931a40f8b05f6dde304390ff => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:22.456305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:22.456593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:22.457198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:22.487032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:22.489005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:22.514606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:22.569709 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2288/4626] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:22.598221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:22.603923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:22.701747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:22.785370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fddc87-0138-4167-8838-d4f0ce435099 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:22.795358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:22.796079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:22.796984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:22.796984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:22.817617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:22.819334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7702ae-94c5-4a89-bb8e-c6323e03bade service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:22.831559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2339/4627] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:23.337593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:23.339354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:23.367837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22ee63-7003-4126-8f0b-8317e56f5564 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:23.395467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:23.396229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:23.397351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:23.400014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] PATH_INFO: `/v3/projects/f6023321fef14d9a9b05fbfa68304359` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:23.405400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6023321fef14d9a9b05fbfa68304359)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:23.418680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:23.519295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6023321fef14d9a9b05fbfa68304359', 'request_id': 'req-276fcd10-ef9e-4569-bbe6-7dbded166f4d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:23.529405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6023321fef14d9a9b05fbfa68304359', 'request_id': 'req-276fcd10-ef9e-4569-bbe6-7dbded166f4d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:23.547345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6023321fef14d9a9b05fbfa68304359', 'request_id': 'req-276fcd10-ef9e-4569-bbe6-7dbded166f4d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:23.567778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] The token cache is being invalidate because project f6023321fef14d9a9b05fbfa68304359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:23.569092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:23.570207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:23.572043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2289/4628] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:23 2022] DELETE /identity/v3/projects/f6023321fef14d9a9b05fbfa68304359 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:23.577377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9562583-0d65-4350-8463-9e9bd59eed05 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:23.577664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9562583-0d65-4350-8463-9e9bd59eed05 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:23.578106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9562583-0d65-4350-8463-9e9bd59eed05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:23.613849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c9562583-0d65-4350-8463-9e9bd59eed05 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:23.619897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9562583-0d65-4350-8463-9e9bd59eed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:23.686884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:23.688433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:23.814836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2340/4629] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:23.823089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:23.823899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:23.824545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:23.852257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:23.858386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:23.889747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcd10-ef9e-4569-bbe6-7dbded166f4d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:23.890057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2341/4630] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:23.894454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:23.916821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:23.918946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:23.919367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:23.919791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] PATH_INFO: `/v3/projects/dde1e0cf8746411a81e4c2d3355715f4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:23.921117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde1e0cf8746411a81e4c2d3355715f4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:23.923607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:23.939796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded70c6a-f3d5-47de-87ed-5e18ae68379b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:23.951998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:23.953026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:23.953026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:23.953532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:23.954022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:23.954985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:23.956750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:23.976951 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2342/4631] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:23.977698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.010698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438150-4c6a-4412-b210-4804ac7b9896 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.015394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde1e0cf8746411a81e4c2d3355715f4', 'request_id': 'req-d57b9e1c-37fe-4734-869b-f658e9c70279'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.018933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:24.025063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde1e0cf8746411a81e4c2d3355715f4', 'request_id': 'req-d57b9e1c-37fe-4734-869b-f658e9c70279'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.037618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde1e0cf8746411a81e4c2d3355715f4', 'request_id': 'req-d57b9e1c-37fe-4734-869b-f658e9c70279'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.058568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] The token cache is being invalidate because project dde1e0cf8746411a81e4c2d3355715f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:24.059631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.060244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.069304 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2290/4632] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:23 2022] DELETE /identity/v3/projects/dde1e0cf8746411a81e4c2d3355715f4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:24.070445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.074225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:24.139024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.143582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.144326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.144782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.145599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.147829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.207274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2343/4633] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:23 2022] POST /identity/v3/projects => generated 365 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:24.216210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.219269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:24.237857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b9e1c-37fe-4734-869b-f658e9c70279 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.256763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:24.298356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6143d-142a-4d7d-89ed-5b6c84b8843d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.304743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.305487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.305840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.306217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.307674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.311929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.315858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.316722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.317292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.317713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] PATH_INFO: `/v3/users/90c26cd616814758a19838999ff22337` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.319073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c26cd616814758a19838999ff22337)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.321018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.349451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2344/4634] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:24 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:24.353773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.366414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38737043-789b-4926-a09c-ba480dfba331 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.372385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.373108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.373440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.373788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.374532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.376304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.390547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2345/4635] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:24.392840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.395451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c26cd616814758a19838999ff22337'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.396131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c26cd616814758a19838999ff22337'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.405911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24099-941a-42eb-a4c7-fedd4ac76336 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.407720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c26cd616814758a19838999ff22337'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.412185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.412984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.413447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.413875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.414726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.416054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c26cd616814758a19838999ff22337'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.416801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.417498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a694d3b5-8cdf-408a-afbb-018ac7d7bd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c26cd616814758a19838999ff22337'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:24.431239 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2346/4636] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:24.434402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.437565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:24.456197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2291/4637] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:24 2022] DELETE /identity/v3/users/90c26cd616814758a19838999ff22337 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:24.467292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb691-379c-4d0d-a78c-6dc4a67d0ec4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.545607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.546424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.546803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.547212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] PATH_INFO: `/v3/projects/5d9195ac8aaa41d7b558ae60158e561b/users/e7c6d5b5e4d7454da96475a419d81b82/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.572810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9195ac8aaa41d7b558ae60158e561b, user_id=e7c6d5b5e4d7454da96475a419d81b82, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.579363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.598220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2347/4638] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:24 2022] PUT /identity/v3/projects/5d9195ac8aaa41d7b558ae60158e561b/users/e7c6d5b5e4d7454da96475a419d81b82/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:24.608528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.608983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.609450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.625990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:24.682722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] MFA Rules not processed for user `e7c6d5b5e4d7454da96475a419d81b82`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:24.684489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:24.800535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2292/4639] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:24.834379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:24.838751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:24.927615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b2217-92a5-4485-9114-97583bb197ab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:24.932698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:24.934277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:24.934729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:24.935006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:24.945734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:24.947585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff12f1e-ff5b-404d-9d5f-5ac2a3ae7d51 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:24.960205 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2348/4640] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:25.281028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:25.282908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:25.436669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151458e8-c08c-4319-9dad-f9d97fa17c3d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:25.445115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:25.446226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:25.447036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:25.447520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] PATH_INFO: `/v3/projects/ee617bd6fe224272a2d8033d3081f462` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:25.449730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee617bd6fe224272a2d8033d3081f462)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:25.452189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:25.543334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee617bd6fe224272a2d8033d3081f462', 'request_id': 'req-bca56f6e-b7fe-42ac-961b-651812985054'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:25.558521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee617bd6fe224272a2d8033d3081f462', 'request_id': 'req-bca56f6e-b7fe-42ac-961b-651812985054'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:25.571850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee617bd6fe224272a2d8033d3081f462', 'request_id': 'req-bca56f6e-b7fe-42ac-961b-651812985054'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:25.590781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] The token cache is being invalidate because project ee617bd6fe224272a2d8033d3081f462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:25.591076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:25.592044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bca56f6e-b7fe-42ac-961b-651812985054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:25.593660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2293/4641] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:25 2022] DELETE /identity/v3/projects/ee617bd6fe224272a2d8033d3081f462 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:25.599517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:25.600187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:25.601336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:25.633643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:25.635289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:25.662177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:25.736830 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2349/4642] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:25.745397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:25.745842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:25.746125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:25.776083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:25.778102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:25.788714 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2294/4643] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:25.793552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:25.810274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc0fd0-3ff8-4733-a53f-ce90494bc64c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:25.819694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:25.820320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:25.820655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:25.820865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:25.821537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:25.821826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:25.824574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:25.837052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2350/4644] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:25.842061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:25.851651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b6c2d-27db-4015-91ea-94ff7a89418b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:25.859283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:25.861457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:25.861903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:25.862677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:25.865572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:25.867817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:25.892873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:25.912343 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2295/4645] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:25 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:25.913495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:25.926948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38300629-a7f8-4d0e-a619-83efbe3e0cd4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.023748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.023748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.023748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.024127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.024534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.027773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.061865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2351/4646] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:25 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:26.063950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.078674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039d7e-dacc-4c71-9800-227139679273 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.085898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.087007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.087593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.088047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.088890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.092948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.114554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2296/4647] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:26.116617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.133594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55c3de-4ed1-47fc-b3a3-8b107ac143a7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.144305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.146185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.146470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.146776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.147481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.149487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bd163-1f18-4958-b1e5-31cdd867f0b6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.177596 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2352/4648] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:26.180161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.215225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804c302-8aeb-4225-b293-1f59b1213e40 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.229155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.232004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.232592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.232876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] PATH_INFO: `/v3/projects/423337c102934affbdd22d522a8ef7e1/users/5cc1de437f194f6eb488450ca384e5d8/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.252328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423337c102934affbdd22d522a8ef7e1, user_id=5cc1de437f194f6eb488450ca384e5d8, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.256404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.275278 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2297/4649] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:26 2022] PUT /identity/v3/projects/423337c102934affbdd22d522a8ef7e1/users/5cc1de437f194f6eb488450ca384e5d8/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:26.287296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.288151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.288643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.338783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] MFA Rules not processed for user `5cc1de437f194f6eb488450ca384e5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:26.341382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:26.461607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2353/4650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:26.462741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.571704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c70867-3b9f-42c3-beda-8c387e3d79b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.581871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.583737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.584139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.584520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.590943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.600130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.667264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.667968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2298/4651] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:26 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:26.689453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c74f5-3f5c-4c07-a1cb-4310d295d9d2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.713769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:26.816205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.816755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.817040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.817400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.820054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.822028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.858377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.869664 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2354/4652] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:26 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:26.870182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175ddf2-1338-460e-a5f1-e8f4c6ab56b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.881439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.882072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.882381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.882655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.883370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.885012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.902059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.908846 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2299/4653] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:26.915264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aede45-768c-4150-9e3b-8fd5fd9d5bbb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:26.920874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:26.921684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:26.922037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:26.923530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] PATH_INFO: `/v3/projects/96e76927e83447fc803201874ade08b1/users/458a694a238b42b380800c64464619cc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:26.946459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e76927e83447fc803201874ade08b1, user_id=458a694a238b42b380800c64464619cc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:26.950899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:26.979608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2355/4654] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:26 2022] PUT /identity/v3/projects/96e76927e83447fc803201874ade08b1/users/458a694a238b42b380800c64464619cc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:26.983924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:26.994243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7e54-612d-4f02-a864-78604f4e9f8b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.079133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.079890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.080322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.080665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.082002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.085581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1866af-a4f6-43ab-8643-b6f035d7d81c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.109387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.120600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2300/4655] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:27.122800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bdefc-9f8f-4325-bd9f-70459d94ffb6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.129593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.132031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.132031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.132031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] PATH_INFO: `/v3/projects/96e76927e83447fc803201874ade08b1/users/458a694a238b42b380800c64464619cc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.134186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e76927e83447fc803201874ade08b1, user_id=458a694a238b42b380800c64464619cc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.137757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.167115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.167115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.167770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.171200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2356/4656] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:27 2022] PUT /identity/v3/projects/96e76927e83447fc803201874ade08b1/users/458a694a238b42b380800c64464619cc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:27.210253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] MFA Rules not processed for user `458a694a238b42b380800c64464619cc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:27.211743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:27.255416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:27.332619 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2301/4657] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:27.341805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.365137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f31aa8-575c-4baa-be3c-238540af7660 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.374657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.376217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.376520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.376849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.377587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.379648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.436576 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2357/4658] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:27 2022] POST /identity/v3/domains => generated 245 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:27.438415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.455869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d002de19-1cb8-44e1-8bda-2c66e2526e43 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.576067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.576067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.576067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.576926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.576926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.579451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.626687 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2302/4659] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:27 2022] POST /identity/v3/projects => generated 410 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:27.627943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.654369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e7fd1-5b31-40cf-b702-265d125488ac tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.763227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.763470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.763655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.764100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.764819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.767752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.816048 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2358/4660] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:27 2022] POST /identity/v3/users => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:27.816999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.834871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d0cf5-938a-447a-9080-15a26697e36b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.846840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.848001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.848479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.849010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:27.850019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:27.854153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0393-351b-4d65-b56c-1fd893595c08 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:27.869895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2303/4661] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:27.875314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:27.896427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898bf66-0e07-468c-9cb9-f9976df2ec51 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:27.941075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:27.942319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:27.942797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:27.943244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/projects/557b05c493a046fdb6b8ceafb6ea3515/users/34206ffa1b3844fea4d892bfbd9362ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.004942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:create_grant(project_id=557b05c493a046fdb6b8ceafb6ea3515, user_id=34206ffa1b3844fea4d892bfbd9362ba, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:28.007698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f866e-95dc-4442-8265-37c52e83d8cf tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:28.029834 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2359/4662] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:27 2022] PUT /identity/v3/projects/557b05c493a046fdb6b8ceafb6ea3515/users/34206ffa1b3844fea4d892bfbd9362ba/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:28.034730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:28.035108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:28.035389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.245032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] MFA Rules not processed for user `34206ffa1b3844fea4d892bfbd9362ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:28.247063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:28.263682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:28.279771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2304/4663] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:28.295438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:28.295983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:28.296435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.362546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] MFA Rules not processed for user `34206ffa1b3844fea4d892bfbd9362ba`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:28.362546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:28.375882 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2360/4664] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3360 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:28.380396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:28.423075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51bf0a-363a-40d6-8802-e4d105e7fc97 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:28.435919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34206ffa1b3844fea4d892bfbd9362ba', 'user_domain_id': '5484eef3284843eb8b8f1d915ca05b70', 'system_scope': None, 'project_id': '557b05c493a046fdb6b8ceafb6ea3515', 'project_domain_id': '5484eef3284843eb8b8f1d915ca05b70', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:28.437146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:28.442166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:28.442846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.480503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:28.488951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:28.524802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2305/4665] 10.222.0.52 () {56 vars in 1284 bytes} [Sat Sep 24 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3360 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:28.531222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:28.569171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:28.681678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0487b09-b925-4759-bbf0-c55c75bae2e9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:28.691268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:28.694005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:28.695492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:28.695787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/users/34206ffa1b3844fea4d892bfbd9362ba` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.701447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:delete_user(user_id=34206ffa1b3844fea4d892bfbd9362ba)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:28.707700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:28.799689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34206ffa1b3844fea4d892bfbd9362ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:28.800483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34206ffa1b3844fea4d892bfbd9362ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:28.809888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34206ffa1b3844fea4d892bfbd9362ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:28.827098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34206ffa1b3844fea4d892bfbd9362ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:28.830688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34206ffa1b3844fea4d892bfbd9362ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:28.856302 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2361/4666] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:28 2022] DELETE /identity/v3/users/34206ffa1b3844fea4d892bfbd9362ba => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:28.858989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:28.873043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216862a-9d89-4d19-95fe-172c040bc1fb tempest-test_project-2095779667 tempest-user-764253164] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:28.950256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:28.950774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:28.950992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:28.951344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/projects/557b05c493a046fdb6b8ceafb6ea3515` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:28.952804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:delete_project(project_id=557b05c493a046fdb6b8ceafb6ea3515)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:28.954533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:29.032822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557b05c493a046fdb6b8ceafb6ea3515', 'request_id': 'req-ef87875d-c65b-4af5-8975-7a465667320b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.050955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557b05c493a046fdb6b8ceafb6ea3515', 'request_id': 'req-ef87875d-c65b-4af5-8975-7a465667320b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.066921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557b05c493a046fdb6b8ceafb6ea3515', 'request_id': 'req-ef87875d-c65b-4af5-8975-7a465667320b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.100631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] The token cache is being invalidate because project 557b05c493a046fdb6b8ceafb6ea3515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:29.100995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.101598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.103422 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2306/4667] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:28 2022] DELETE /identity/v3/projects/557b05c493a046fdb6b8ceafb6ea3515 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:29.107376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:29.110161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:29.141812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1b41e-9fa6-46c9-b7a1-af8c694ca192 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:29.252861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:29.253584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:29.254007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:29.254119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/domains/5484eef3284843eb8b8f1d915ca05b70` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:29.257826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=5484eef3284843eb8b8f1d915ca05b70)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:29.259138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:29.311676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.312852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.356389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:29.386698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.389894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.398116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2362/4668] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:09:29 2022] PATCH /identity/v3/domains/5484eef3284843eb8b8f1d915ca05b70 => generated 246 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:29.402408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:29.404252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:29.440269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87875d-c65b-4af5-8975-7a465667320b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:29.640249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458a694a238b42b380800c64464619cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96e76927e83447fc803201874ade08b1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:29.640915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:29.641203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:29.641474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] PATH_INFO: `/v3/domains/5484eef3284843eb8b8f1d915ca05b70` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:29.679224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5484eef3284843eb8b8f1d915ca05b70)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:29.680861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:29.683888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:29.753737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.775940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.884686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.892919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.906352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.938391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] The token cache is being invalidate because project 5484eef3284843eb8b8f1d915ca05b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:29.938706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.941877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.955209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.956283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.965595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4 tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5484eef3284843eb8b8f1d915ca05b70', 'request_id': 'req-4ea3c6e1-73e8-4068-8344-a3ce6586f5b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:29.975765 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2307/4669] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:29 2022] DELETE /identity/v3/domains/5484eef3284843eb8b8f1d915ca05b70 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:29.981172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:29.982629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:30.096949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d5c6a-a787-47fd-b0cd-6291711f589b tempest-TestDefaultProjectId-1495350372 tempest-TestDefaultProjectId-1495350372-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:30.217828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:30.219824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:30.223618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:30.225419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] PATH_INFO: `/v3/users/5cc1de437f194f6eb488450ca384e5d8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:30.229369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc1de437f194f6eb488450ca384e5d8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:30.234757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:30.330099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc1de437f194f6eb488450ca384e5d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:30.331612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc1de437f194f6eb488450ca384e5d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:30.347201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc1de437f194f6eb488450ca384e5d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:30.387559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc1de437f194f6eb488450ca384e5d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:30.391576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc1de437f194f6eb488450ca384e5d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:30.430088 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2363/4670] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:29 2022] DELETE /identity/v3/users/5cc1de437f194f6eb488450ca384e5d8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:30.431393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:30.431798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:30.431958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:30.472662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:30.473912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:30.624927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2308/4671] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:30.656876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:30.658393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:30.721728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:30.800029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba14e7-472a-4a08-b10f-8a439a994a70 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:30.806486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:30.808784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:30.809058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:30.810197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:30.829616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:30.833128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:30.862296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2364/4672] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:32.672649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:32.674829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:32.703392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70f64-a14f-4886-b89e-6b241888ee60 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:32.714728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:32.715583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:32.716042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:32.716462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] PATH_INFO: `/v3/projects/423337c102934affbdd22d522a8ef7e1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:32.718365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423337c102934affbdd22d522a8ef7e1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:32.722868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:32.842618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:32.874660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423337c102934affbdd22d522a8ef7e1', 'request_id': 'req-1c00f776-00a3-4ced-a04f-02c56bd17c50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:32.885708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423337c102934affbdd22d522a8ef7e1', 'request_id': 'req-1c00f776-00a3-4ced-a04f-02c56bd17c50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:32.929535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423337c102934affbdd22d522a8ef7e1', 'request_id': 'req-1c00f776-00a3-4ced-a04f-02c56bd17c50'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:32.957466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] The token cache is being invalidate because project 423337c102934affbdd22d522a8ef7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:32.957774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:32.958456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:32.964415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2309/4673] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:32 2022] DELETE /identity/v3/projects/423337c102934affbdd22d522a8ef7e1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:32.965864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:32.978118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d92b-1ff3-4645-87e5-c7d5556cefa2 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:33.140572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:33.141248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:33.141525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:33.141801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] PATH_INFO: `/v3/users/458a694a238b42b380800c64464619cc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:33.143023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=458a694a238b42b380800c64464619cc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:33.145683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:33.325222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458a694a238b42b380800c64464619cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:33.328358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458a694a238b42b380800c64464619cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:33.349945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458a694a238b42b380800c64464619cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:33.384508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458a694a238b42b380800c64464619cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:33.385382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d96d36c9-56e6-4790-ad89-f7f71cede0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458a694a238b42b380800c64464619cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:33.421490 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2365/4674] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:32 2022] DELETE /identity/v3/users/458a694a238b42b380800c64464619cc => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:34.533598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:34.536983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:34.559550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00f776-00a3-4ced-a04f-02c56bd17c50 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:34.663421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:34.664066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:34.664346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:34.664616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] PATH_INFO: `/v3/projects/96e76927e83447fc803201874ade08b1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:34.665928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e76927e83447fc803201874ade08b1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:34.668667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:34.733537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e76927e83447fc803201874ade08b1', 'request_id': 'req-e23215ae-450f-4431-b21b-564d3af267aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:34.740864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e76927e83447fc803201874ade08b1', 'request_id': 'req-e23215ae-450f-4431-b21b-564d3af267aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:34.750801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e76927e83447fc803201874ade08b1', 'request_id': 'req-e23215ae-450f-4431-b21b-564d3af267aa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:34.764811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] The token cache is being invalidate because project 96e76927e83447fc803201874ade08b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:34.765030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:34.765652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e23215ae-450f-4431-b21b-564d3af267aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:34.770569 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2310/4675] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:34 2022] DELETE /identity/v3/projects/96e76927e83447fc803201874ade08b1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:34.772300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:34.772801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:34.772894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:34.799913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:34.801197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:34.855089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:34.875103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2366/4676] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:34.882769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:34.883060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:34.883341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:34.914974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:34.917350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:34.926069 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2311/4677] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:34.930327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:34.943133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea531-ab4f-4552-970a-47eacb37eef2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:34.950243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:34.952116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:34.952116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:34.952116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:34.952906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:34.953317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:34.956622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:34.977187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2367/4678] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:34.981650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:34.992420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64120441-344c-4de9-9d51-6a0575a37a8a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:34.998340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:34.998996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:34.999278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:34.999671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.000240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.002334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.051300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2312/4679] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:34 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:35.056598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.080759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d63ab-375b-4f94-99f7-556994ce7bc4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:35.167472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:35.168370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.168865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.169437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.170385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.172967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.204628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2368/4680] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:35 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:35.207857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.220013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189c9c-5892-4b09-ab31-d61bf4d06093 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:35.225123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:35.225728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.226014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.226288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.227258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.228882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.244050 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2313/4681] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:35.244784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.257125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6d747-1955-4026-97ad-fccc6ca8126c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:35.265097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:35.266114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.266395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.267590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.268322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.271653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34740-9958-4a20-b4af-8d8fb071c9f7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.290664 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2369/4682] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:35.295612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.320028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217cf82-2e64-4b07-b2c3-3ddf4df0a1e2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:35.325625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:35.326797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.328206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.328609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] PATH_INFO: `/v3/projects/5bb36de73f044b7e959bf6ec10c56b24/users/dc698a4b6e204b7fbe196c76f219971a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.357223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb36de73f044b7e959bf6ec10c56b24, user_id=dc698a4b6e204b7fbe196c76f219971a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.361982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.375239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:35.397016 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2314/4683] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:35 2022] PUT /identity/v3/projects/5bb36de73f044b7e959bf6ec10c56b24/users/dc698a4b6e204b7fbe196c76f219971a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:35.399508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.400152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.400674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.496322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] MFA Rules not processed for user `dc698a4b6e204b7fbe196c76f219971a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:35.497838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:35.690166 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2370/4684] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:35.693392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.848486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17316-a157-4c53-94ec-f9e7f1161fd2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:35.858024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:35.858986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:35.859476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:35.859988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:35.860948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:35.863182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:35.912902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2315/4685] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:35 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:35.913862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:35.928724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b70a-aec9-42cb-8583-cbeb04d558d2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.003761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:36.054353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.055412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.055936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.056428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.059717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.063333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.126854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2371/4686] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:35 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:36.130239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.143526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdd1e0-5dc7-423e-9bce-072f0f135b70 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.149339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.149830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.150216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.152332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.153078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.154877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.169658 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2316/4687] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:36.171557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.182001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6767b-0a48-4155-a33e-759ff14a555a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.187209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.187769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.188394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.188676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] PATH_INFO: `/v3/projects/dc93695013934bf8b244aaf9dee7bc06/users/daf0a04aaa714977abad924150223a99/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.209805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc93695013934bf8b244aaf9dee7bc06, user_id=daf0a04aaa714977abad924150223a99, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.215598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2f358-fcbf-418e-854b-8db149ea8ebd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.237089 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2372/4688] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:36 2022] PUT /identity/v3/projects/dc93695013934bf8b244aaf9dee7bc06/users/daf0a04aaa714977abad924150223a99/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:36.238835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.249765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa44d6-d12b-442c-8ced-21619f17e586 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.309807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.309807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.310044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.343991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.344742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.345024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.345296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.346071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.347745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.347870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] MFA Rules not processed for user `a22830d73cb043b8b9c2e070afa7af55`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:36.349046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:36.359927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2317/4689] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:36.364415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.375338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a8d2-1869-4302-8d71-6ddcb6c71e14 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.380376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.381166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.381676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.381893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] PATH_INFO: `/v3/projects/dc93695013934bf8b244aaf9dee7bc06/users/daf0a04aaa714977abad924150223a99/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.384048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc93695013934bf8b244aaf9dee7bc06, user_id=daf0a04aaa714977abad924150223a99, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.387918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6c759-bb72-4ca9-8f56-1ef88a408c66 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.400059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2318/4690] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:36 2022] PUT /identity/v3/projects/dc93695013934bf8b244aaf9dee7bc06/users/daf0a04aaa714977abad924150223a99/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:36.412787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.413073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.413490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.418540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2373/4691] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:36.451924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] MFA Rules not processed for user `daf0a04aaa714977abad924150223a99`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:36.453116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:36.508669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:36.561685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2319/4692] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:36.566230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.663893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc4396-bf8d-40fb-acb0-aa3bc3881bb5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.670068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.670556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.670850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.671125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.671777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.675033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.723076 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2374/4693] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:36 2022] POST /identity/v3/projects => generated 360 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:36.724622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.739941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d19dd-a5b6-4b6d-b721-8fd3a54d9643 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.859278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.860364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.860364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.860563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.861268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.863230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.869772 ncdv-master-1 devstack@keystone.service[99133]: WARNING keystone.server.flask.common [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 21:09:36.906648 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2320/4694] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:36 2022] POST /identity/v3/users => generated 416 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:36.910970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.926530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd1b46-bc81-472e-902e-fc571d69ed51 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.936545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.937204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:36.937670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:36.937808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/users/a6972433a8454949850a41b98950c15a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:36.960903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_user(user_id=a6972433a8454949850a41b98950c15a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:36.965514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:36.967672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2375/4695] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:09:36 2022] GET /identity/v3/users/a6972433a8454949850a41b98950c15a => generated 416 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:36.972574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:36.987847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fae03c-689d-4658-a343-0e8437398c94 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:36.999113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:36.999766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.000047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.000325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/users/a6972433a8454949850a41b98950c15a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.004095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a6972433a8454949850a41b98950c15a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.013997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.110644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6972433a8454949850a41b98950c15a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.111267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6972433a8454949850a41b98950c15a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.129989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6972433a8454949850a41b98950c15a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.142404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6972433a8454949850a41b98950c15a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.143272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6972433a8454949850a41b98950c15a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.182120 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2321/4696] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:36 2022] DELETE /identity/v3/users/a6972433a8454949850a41b98950c15a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:37.186489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.188329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:37.212151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf84224-b7c6-48ae-ab7b-be52da90e329 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:37.240621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:37.297612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:37.298524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.299034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.299497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f44fd20ac6b3414b8ffe3069f9787f47` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.300911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f44fd20ac6b3414b8ffe3069f9787f47)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.304069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.386098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44fd20ac6b3414b8ffe3069f9787f47', 'request_id': 'req-2780fbef-0291-46b7-bf31-a8253f9df466'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.392158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44fd20ac6b3414b8ffe3069f9787f47', 'request_id': 'req-2780fbef-0291-46b7-bf31-a8253f9df466'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.400140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44fd20ac6b3414b8ffe3069f9787f47', 'request_id': 'req-2780fbef-0291-46b7-bf31-a8253f9df466'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.414419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project f44fd20ac6b3414b8ffe3069f9787f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:37.414890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.416399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.423330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2376/4697] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:37 2022] DELETE /identity/v3/projects/f44fd20ac6b3414b8ffe3069f9787f47 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:37.425066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.436722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e230-3c26-46b6-b818-178b960256e9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:37.486412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:37.516619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:37.517419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.517780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.518129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.518880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.520667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.539110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2322/4698] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:37 2022] POST /identity/v3/projects => generated 347 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:37.544087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.545265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:37.562189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780fbef-0291-46b7-bf31-a8253f9df466 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:37.652016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:37.652636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.653082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.653357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.654016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:37.654318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.657578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.679821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2377/4699] 10.222.0.52 () {58 vars in 1160 bytes} [Sat Sep 24 21:09:37 2022] GET /identity/v3/projects?is_domain=True => generated 1019 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:37.685664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.699959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1538c-62d2-4ab6-ad6b-a59029939879 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:37.705259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:37.705895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.706171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.707832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.707832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.709217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.730732 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2323/4700] 10.222.0.52 () {58 vars in 1128 bytes} [Sat Sep 24 21:09:37 2022] GET /identity/v3/domains => generated 828 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:37.734581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.753478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eccc01-a996-4f2d-942a-84292af75e32 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:37.771639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:37.772287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:37.773459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:37.773750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f7fe3bdc376c4206887b711fb5cc3c57` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:37.775452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_project(project_id=f7fe3bdc376c4206887b711fb5cc3c57)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:37.776732 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:37.776732 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:09:37.776732 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:09:37.783992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:37.798881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.799960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.863701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.865665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:37.869183 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2378/4701] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:09:37 2022] PATCH /identity/v3/projects/f7fe3bdc376c4206887b711fb5cc3c57 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:37.874159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:37.878518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:37.897746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ff1d-837e-4faa-aeb6-d8c191cbb38b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.010213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.011003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.011312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.011565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f7fe3bdc376c4206887b711fb5cc3c57` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.025144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7fe3bdc376c4206887b711fb5cc3c57)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.027585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.052709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.065797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.128845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.134757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.144380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.161824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project f7fe3bdc376c4206887b711fb5cc3c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:38.161824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.162795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.174071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.174804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.178995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f7fe3bdc376c4206887b711fb5cc3c57', 'request_id': 'req-55d8cd27-7271-4677-8721-310b08bad2a6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.181675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:38.189957 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2324/4702] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:37 2022] DELETE /identity/v3/projects/f7fe3bdc376c4206887b711fb5cc3c57 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:38.195409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.210605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ad460-fda4-4aca-b9a1-a0db534414f4 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.238875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:38.320466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.321384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.321816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.322256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.323303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.326095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.365119 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2379/4703] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:38 2022] POST /identity/v3/projects => generated 359 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:38.369994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.372196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:38.384936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8cd27-7271-4677-8721-310b08bad2a6 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.467767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.468402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.468703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.468980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/3117cdb3ef234d22baa6c01558dc1b3f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.470791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=3117cdb3ef234d22baa6c01558dc1b3f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.472483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.474542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2325/4704] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:38 2022] GET /identity/v3/projects/3117cdb3ef234d22baa6c01558dc1b3f => generated 359 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:38.479299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.493292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d168540-5cbf-4338-9580-0d1078c38cd1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.498538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.500502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.500788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.502330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/3117cdb3ef234d22baa6c01558dc1b3f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.502543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3117cdb3ef234d22baa6c01558dc1b3f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.506124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.573347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3117cdb3ef234d22baa6c01558dc1b3f', 'request_id': 'req-eb0c7c1b-a373-438e-9958-1dfff5493eae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.579472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3117cdb3ef234d22baa6c01558dc1b3f', 'request_id': 'req-eb0c7c1b-a373-438e-9958-1dfff5493eae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.587041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3117cdb3ef234d22baa6c01558dc1b3f', 'request_id': 'req-eb0c7c1b-a373-438e-9958-1dfff5493eae'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.602180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 3117cdb3ef234d22baa6c01558dc1b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:38.602463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.603839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.610391 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2380/4705] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:38 2022] DELETE /identity/v3/projects/3117cdb3ef234d22baa6c01558dc1b3f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:38.613632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.615755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:38.631397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0f82-eec9-4348-9811-659c71de5af7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.732821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.734180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.734670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.735099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.736178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.739703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.780357 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2326/4706] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:38 2022] POST /identity/v3/projects => generated 360 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:38.784626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.796568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7c1b-a373-438e-9958-1dfff5493eae tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.880584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.881671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.882130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.882587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/d7559c083e814dbe9de72aeeb941015e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.883944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=d7559c083e814dbe9de72aeeb941015e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.886676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.893266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:38.894292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2381/4707] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:38 2022] GET /identity/v3/projects/d7559c083e814dbe9de72aeeb941015e => generated 360 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:38.906841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac464c-934f-4a65-8ae7-2d9b94e4a1d0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:38.912657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:38.913505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:38.913656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:38.913847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/d7559c083e814dbe9de72aeeb941015e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:38.915015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d7559c083e814dbe9de72aeeb941015e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:38.917709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:38.977993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7559c083e814dbe9de72aeeb941015e', 'request_id': 'req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.984965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7559c083e814dbe9de72aeeb941015e', 'request_id': 'req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:38.996311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7559c083e814dbe9de72aeeb941015e', 'request_id': 'req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.008967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project d7559c083e814dbe9de72aeeb941015e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:39.009261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.010022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.016937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:39.017435 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2327/4708] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:38 2022] DELETE /identity/v3/projects/d7559c083e814dbe9de72aeeb941015e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:39.018993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:39.035526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f20ca-fee3-44eb-9d91-179837ff318f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:39.070634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:39.172282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:39.173181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:39.173479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:39.174654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:39.175504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:39.178157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:39.228009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:39.230531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2382/4709] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:09:39 2022] POST /identity/v3/projects => generated 345 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:39.255471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcddab3-cc41-4e31-8580-d2b1cd9f2d7d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:39.279511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:39.424166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:39.425348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:39.425749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:39.426140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/0e7a4de6adbd48fdba749fcc84147482` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:39.433325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=0e7a4de6adbd48fdba749fcc84147482)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:39.435610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:39.445308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2328/4710] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:39 2022] GET /identity/v3/projects/0e7a4de6adbd48fdba749fcc84147482 => generated 345 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:09:39.451005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:39.462360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993650c-769b-40bf-bb05-64411c5e95e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:39.486025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:39.486960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:39.487345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:39.488109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/0e7a4de6adbd48fdba749fcc84147482` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:39.489139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e7a4de6adbd48fdba749fcc84147482)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:39.491587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:39.592859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7a4de6adbd48fdba749fcc84147482', 'request_id': 'req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.603321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7a4de6adbd48fdba749fcc84147482', 'request_id': 'req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.655162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7a4de6adbd48fdba749fcc84147482', 'request_id': 'req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.687428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 0e7a4de6adbd48fdba749fcc84147482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:39.688015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.689098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:39.691191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2383/4711] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:39 2022] DELETE /identity/v3/projects/0e7a4de6adbd48fdba749fcc84147482 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:39.695910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:39.697890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:39.729693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fa9b2-9d48-48b9-9c54-2ac15de3aa5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:39.802057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:39.802834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:39.803327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:39.803763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:39.804805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:39.808450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:39.833125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:39.839306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2329/4712] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:39 2022] POST /identity/v3/domains => generated 276 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:39.847943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227cf4-7881-48a7-a3f8-e4b953db3ff5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:39.971474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:39.972270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:39.972590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:39.973126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:39.974490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:39.979848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:40.085222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2384/4713] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:39 2022] POST /identity/v3/projects => generated 404 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:40.086475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:40.097730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbcb18-a4ea-47fe-94e7-f96da19edfa2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:40.173676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:40.174407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:40.174814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:40.175141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/06a35f693e344c39898b8c78977909c6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:40.176412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=06a35f693e344c39898b8c78977909c6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:40.178751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:40.180812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2330/4714] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:40 2022] GET /identity/v3/projects/06a35f693e344c39898b8c78977909c6 => generated 404 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:40.185540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:40.195739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be12a-5cc6-40a3-b905-8460192e0c1f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:40.200743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:40.201425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:40.201662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:40.201933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/06a35f693e344c39898b8c78977909c6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:40.203160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06a35f693e344c39898b8c78977909c6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:40.205082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:40.212046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:40.261861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a35f693e344c39898b8c78977909c6', 'request_id': 'req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.271276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a35f693e344c39898b8c78977909c6', 'request_id': 'req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.278323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a35f693e344c39898b8c78977909c6', 'request_id': 'req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.295061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 06a35f693e344c39898b8c78977909c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:40.295455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.296983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.302584 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2385/4715] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:40 2022] DELETE /identity/v3/projects/06a35f693e344c39898b8c78977909c6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:40.303452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:40.305574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:40.322968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36381d6b-231f-42bd-acae-fc49c588b3ce tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:40.387995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:40.438218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:40.442033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:40.442324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:40.443205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/882e7e29f2f64170956b3259fc0891b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:40.445424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=882e7e29f2f64170956b3259fc0891b6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:40.450720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:40.513743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.514687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.598585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.600833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.614553 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2331/4716] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:09:40 2022] PATCH /identity/v3/domains/882e7e29f2f64170956b3259fc0891b6 => generated 277 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:40.618891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:40.633815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70f7f2-9422-4d0d-8e44-64a6014caeb1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:40.732873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:40.734345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:40.734881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:40.735382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/882e7e29f2f64170956b3259fc0891b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:40.751052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=882e7e29f2f64170956b3259fc0891b6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:40.753630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:40.784287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.794368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '882e7e29f2f64170956b3259fc0891b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.853355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.860878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.869165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.871363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:40.903367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 882e7e29f2f64170956b3259fc0891b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:40.903646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.904548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.914346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.919407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.920244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '882e7e29f2f64170956b3259fc0891b6', 'request_id': 'req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:40.931475 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2386/4717] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:40 2022] DELETE /identity/v3/domains/882e7e29f2f64170956b3259fc0891b6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:40.932788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:40.934853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:40.957034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8802f5-822e-4f27-9a6f-b020afe2a516 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:41.050621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:41.051562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:41.052171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:41.052622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:41.053440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:41.055541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:41.082921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2332/4718] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:40 2022] POST /identity/v3/domains => generated 277 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:41.087814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:41.090394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:41.106280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa8702-49d6-42b2-aaab-5e268a2b15d8 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:41.243301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:41.244287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:41.245276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:41.247032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:41.247868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:41.254543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:41.323909 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2387/4719] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:41 2022] POST /identity/v3/projects => generated 409 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:41.328104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:41.342577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840aabc-2721-4f3d-9517-f29d90b420d5 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:41.484624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:41.485360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:41.485676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:41.485952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:41.488515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:41.496540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b426b-572f-42fc-b1cc-f9dae4d638e1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:41.601883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2333/4720] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:41 2022] POST /identity/v3/projects => generated 405 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:41.605885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:41.625708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220d25e-ae45-4654-a972-a9a9c41a43f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:41.815218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:41.817851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:41.819144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:41.819961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/e8161e0e6220411881dab789e8814e77` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:41.827633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e8161e0e6220411881dab789e8814e77)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:41.834380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:41.850539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:41.850539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:41.850836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:41.914189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] MFA Rules not processed for user `ca78cf6d34e84f2cb5cb254c5d8af1da`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:41.915911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:41.959262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:41.965185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8161e0e6220411881dab789e8814e77', 'request_id': 'req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:41.966837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:41.980392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8161e0e6220411881dab789e8814e77', 'request_id': 'req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:41.994284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8161e0e6220411881dab789e8814e77', 'request_id': 'req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.021129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project e8161e0e6220411881dab789e8814e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:42.021725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.023162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.025484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2388/4721] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:41 2022] DELETE /identity/v3/projects/e8161e0e6220411881dab789e8814e77 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:42.029863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:42.033632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:42.044880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2334/4722] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:42.170934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f84aa5-a4f6-4c5d-a5f2-0083870b585f tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:42.177174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:42.177817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:42.178095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:42.178368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/6be7163d275c488fa3b07c32222d5322` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:42.179889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6be7163d275c488fa3b07c32222d5322)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:42.182170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:42.265067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be7163d275c488fa3b07c32222d5322', 'request_id': 'req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.272238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be7163d275c488fa3b07c32222d5322', 'request_id': 'req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.290470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be7163d275c488fa3b07c32222d5322', 'request_id': 'req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.321926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 6be7163d275c488fa3b07c32222d5322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:42.324116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.326476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.331564 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2389/4723] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:42 2022] DELETE /identity/v3/projects/6be7163d275c488fa3b07c32222d5322 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:42.335463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:42.352910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5169-1f12-4c23-baea-aca59284d147 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:42.560088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:42.561726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:42.562237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:42.563268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/4487b48c800f468e813bf49196fb8554` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:42.565985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=4487b48c800f468e813bf49196fb8554)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:42.569049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:42.610000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.612015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.733916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.735011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:42.741900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2335/4724] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:09:42 2022] PATCH /identity/v3/domains/4487b48c800f468e813bf49196fb8554 => generated 278 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:42.746567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:42.748052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:42.787406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41e196-ec9f-4dc0-8c69-f1d4ac990221 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:42.934683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:42.935316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:42.936136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:42.936136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/4487b48c800f468e813bf49196fb8554` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:42.967717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=4487b48c800f468e813bf49196fb8554)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:42.969800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:42.971775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:43.012300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.019611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '4487b48c800f468e813bf49196fb8554'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.148496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.160012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.199730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.234080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 4487b48c800f468e813bf49196fb8554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:43.235108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.236210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.297978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.322247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.324917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '4487b48c800f468e813bf49196fb8554', 'request_id': 'req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:43.334544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2390/4725] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:42 2022] DELETE /identity/v3/domains/4487b48c800f468e813bf49196fb8554 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:43.340214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:43.342304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:43.373411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa1ddd-5c0a-45b2-8189-13682ac9f37d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:43.519347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:43.520038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:43.520733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:43.520733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:43.521662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:43.524574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:43.586914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2336/4726] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:43 2022] POST /identity/v3/projects => generated 377 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:43.588159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:43.590911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:43.606467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea53b-c4ec-47cf-9d27-657866ef2df0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:43.690696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:43.691359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:43.691640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:43.691919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/65e21884d7114dff9b354a543eb6c754` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:43.693202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=65e21884d7114dff9b354a543eb6c754)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:43.695525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:43.703819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:43.704013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2391/4727] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:43 2022] GET /identity/v3/projects/65e21884d7114dff9b354a543eb6c754 => generated 377 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:43.731956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d02b4c-b944-4c46-9b13-45e95eb57ed7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:43.737531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:43.738156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:43.739408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:43.739702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:43.740419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:43.742553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:43.782050 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2337/4728] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:09:43 2022] GET /identity/v3/projects => generated 6584 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:43.787410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:43.801660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c27e2-0f86-453e-97e4-40b3d8a1c10d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:43.813068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:43.813725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:43.814007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:43.814286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/65e21884d7114dff9b354a543eb6c754` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:43.820875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=65e21884d7114dff9b354a543eb6c754)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:43.845997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:43.993042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:44.006031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65e21884d7114dff9b354a543eb6c754', 'request_id': 'req-8a08d99c-597b-4720-8544-a92a87e6ea5c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.022386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65e21884d7114dff9b354a543eb6c754', 'request_id': 'req-8a08d99c-597b-4720-8544-a92a87e6ea5c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.036016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65e21884d7114dff9b354a543eb6c754', 'request_id': 'req-8a08d99c-597b-4720-8544-a92a87e6ea5c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.066665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 65e21884d7114dff9b354a543eb6c754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:44.067808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.070900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.079594 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2392/4729] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:43 2022] DELETE /identity/v3/projects/65e21884d7114dff9b354a543eb6c754 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:44.081203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:44.086029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:44.106429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:44.146283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b0428-4ff1-4d2c-9202-8a9670f7328b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:44.427475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:44.428899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:44.429343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:44.431265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:44.437326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:44.450038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:44.542749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:44.546640 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2338/4730] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:09:44 2022] POST /identity/v3/projects => generated 348 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:44.556560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08d99c-597b-4720-8544-a92a87e6ea5c tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:44.648283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:44.649213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:44.649725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:44.650184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/e5ab85edff614486aadbf1233075648c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:44.652831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_project(project_id=e5ab85edff614486aadbf1233075648c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:44.655106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:44.735832 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2393/4731] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:09:44 2022] PATCH /identity/v3/projects/e5ab85edff614486aadbf1233075648c => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:44.740324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:44.753924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150623e-461d-4588-9c28-3e52d1748bdc tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:44.760200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:44.762280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:44.762765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:44.763321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/e5ab85edff614486aadbf1233075648c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:44.777665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=e5ab85edff614486aadbf1233075648c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:44.780846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:44.787388 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2339/4732] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:44 2022] GET /identity/v3/projects/e5ab85edff614486aadbf1233075648c => generated 349 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:44.788165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:44.800136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8aec4-2a08-48bd-871b-a72bb6f277f2 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:44.807742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:44.807742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:44.808166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:44.808629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/e5ab85edff614486aadbf1233075648c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:44.810072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e5ab85edff614486aadbf1233075648c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:44.812267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:44.882293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ab85edff614486aadbf1233075648c', 'request_id': 'req-0d15443b-1b31-49a8-841d-9d78b06dd8b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.890224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ab85edff614486aadbf1233075648c', 'request_id': 'req-0d15443b-1b31-49a8-841d-9d78b06dd8b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.898890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ab85edff614486aadbf1233075648c', 'request_id': 'req-0d15443b-1b31-49a8-841d-9d78b06dd8b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.922369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project e5ab85edff614486aadbf1233075648c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:44.922749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.923850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:44.931412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:44.932455 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2394/4733] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:44 2022] DELETE /identity/v3/projects/e5ab85edff614486aadbf1233075648c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:44.933899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:44.947658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1b7a0-d07d-4e98-81b6-66a8afb23bc3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:45.045212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:45.045984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:45.046369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:45.046928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:45.047918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:45.053171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:45.101427 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2340/4734] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:44 2022] POST /identity/v3/projects => generated 360 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:45.106395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:45.123997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15443b-1b31-49a8-841d-9d78b06dd8b0 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:45.176741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:45.231461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:45.232412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:45.232842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:45.233425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/313467eff25349ad8500090a7879c60f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:45.234838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_project(project_id=313467eff25349ad8500090a7879c60f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:45.237168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:45.313077 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2395/4735] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:09:45 2022] PATCH /identity/v3/projects/313467eff25349ad8500090a7879c60f => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:45.317933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:45.338928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e865-7010-489e-8c92-7f5b9dff15c1 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:45.352670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:45.353600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:45.355785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:45.357615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/313467eff25349ad8500090a7879c60f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:45.387391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=313467eff25349ad8500090a7879c60f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:45.393869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:45.405826 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2341/4736] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:45 2022] GET /identity/v3/projects/313467eff25349ad8500090a7879c60f => generated 359 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:45.407560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:45.425178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d958368-4b16-4bad-8d3e-fbcda5a46195 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:45.438689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:45.440256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:45.440933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:45.441872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/313467eff25349ad8500090a7879c60f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:45.444856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=313467eff25349ad8500090a7879c60f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:45.455837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:45.597194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313467eff25349ad8500090a7879c60f', 'request_id': 'req-ce98d160-b78f-4381-b690-8a4b287e4bd7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:45.609263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313467eff25349ad8500090a7879c60f', 'request_id': 'req-ce98d160-b78f-4381-b690-8a4b287e4bd7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:45.624735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313467eff25349ad8500090a7879c60f', 'request_id': 'req-ce98d160-b78f-4381-b690-8a4b287e4bd7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:45.681245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project 313467eff25349ad8500090a7879c60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:45.681626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:45.684056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:45.686874 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2396/4737] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:45 2022] DELETE /identity/v3/projects/313467eff25349ad8500090a7879c60f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:45.692539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:45.698398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:45.731417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb72667-0757-45bb-9f51-9c2aaf5a57e3 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:45.845312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:45.875789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:45.876569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:45.877010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:45.877383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:45.878143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:45.880854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:45.944558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2342/4738] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:45 2022] POST /identity/v3/projects => generated 352 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:45.948545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:45.964087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98d160-b78f-4381-b690-8a4b287e4bd7 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:46.085235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:46.085882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:46.086161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:46.086584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f4f529d57cdd4974bce07045c9d3a903` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:46.087928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:update_project(project_id=f4f529d57cdd4974bce07045c9d3a903)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:46.090818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:46.143801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:UPDATE=2,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:46.181467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:46.184856 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2397/4739] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:09:45 2022] PATCH /identity/v3/projects/f4f529d57cdd4974bce07045c9d3a903 => generated 369 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:46.224359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ca4e6-3d5c-44d7-b20a-761bdd85524d tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:46.271944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:46.273265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:46.273980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:46.275519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f4f529d57cdd4974bce07045c9d3a903` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:46.319566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:get_project(project_id=f4f529d57cdd4974bce07045c9d3a903)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:46.322122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:46.326900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2343/4740] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:46 2022] GET /identity/v3/projects/f4f529d57cdd4974bce07045c9d3a903 => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:46.332668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:46.351331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e08da3-92a1-48bb-bb18-97d8f6faa945 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:46.366274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:46.367104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:46.367494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:46.367879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/projects/f4f529d57cdd4974bce07045c9d3a903` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:46.372813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f4f529d57cdd4974bce07045c9d3a903)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:46.374648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:46.499329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f529d57cdd4974bce07045c9d3a903', 'request_id': 'req-00c6a3f4-a14c-4282-839c-658c83b37a5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:46.535964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f529d57cdd4974bce07045c9d3a903', 'request_id': 'req-00c6a3f4-a14c-4282-839c-658c83b37a5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:46.554733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f529d57cdd4974bce07045c9d3a903', 'request_id': 'req-00c6a3f4-a14c-4282-839c-658c83b37a5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:46.579266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] The token cache is being invalidate because project f4f529d57cdd4974bce07045c9d3a903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:46.579885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:46.580828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:46.582844 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2398/4741] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:46 2022] DELETE /identity/v3/projects/f4f529d57cdd4974bce07045c9d3a903 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:46.588631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:46.590683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:46.611840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e262305-36f5-4416-b2e9-d3a7e686403b tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:46.747131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:46.748901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:46.749413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:46.750042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/4487b48c800f468e813bf49196fb8554` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:46.763352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Could not find domain: 4487b48c800f468e813bf49196fb8554.: keystone.exception.DomainNotFound: Could not find domain: 4487b48c800f468e813bf49196fb8554. Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4487b48c800f468e813bf49196fb8554. Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:09:46.775811 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:09:46.777042 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 4487b48c800f468e813bf49196fb8554. Sep 24 21:09:46.778743 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:46.782670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:46.790277 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2344/4742] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:09:46 2022] PATCH /identity/v3/domains/4487b48c800f468e813bf49196fb8554 => generated 112 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:09:46.796079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6a3f4-a14c-4282-839c-658c83b37a5e tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:46.808600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf0a04aaa714977abad924150223a99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc93695013934bf8b244aaf9dee7bc06', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:46.809248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:46.814162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:46.814522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] PATH_INFO: `/v3/domains/882e7e29f2f64170956b3259fc0891b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:46.834155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-95615e65-6904-4a17-9afe-9ee9f8113fdf tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Could not find domain: 882e7e29f2f64170956b3259fc0891b6.: keystone.exception.DomainNotFound: Could not find domain: 882e7e29f2f64170956b3259fc0891b6. Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 882e7e29f2f64170956b3259fc0891b6. Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:09:46.847814 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:09:46.850950 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 882e7e29f2f64170956b3259fc0891b6. Sep 24 21:09:46.851972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:09:46.853514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2399/4743] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:09:46 2022] PATCH /identity/v3/domains/882e7e29f2f64170956b3259fc0891b6 => generated 112 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:09:46.858207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:46.861336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:46.997551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db76884-3477-4b13-85e2-464ab78560f9 tempest-ProjectsTestJSON-679263194 tempest-ProjectsTestJSON-679263194-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:47.011286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:47.012521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:47.013029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:47.013482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] PATH_INFO: `/v3/users/dc698a4b6e204b7fbe196c76f219971a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:47.016763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc698a4b6e204b7fbe196c76f219971a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:47.020396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:47.121332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc698a4b6e204b7fbe196c76f219971a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.125188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc698a4b6e204b7fbe196c76f219971a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.160775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc698a4b6e204b7fbe196c76f219971a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.203525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:47.204743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc698a4b6e204b7fbe196c76f219971a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.205647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc698a4b6e204b7fbe196c76f219971a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.234318 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2345/4744] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:46 2022] DELETE /identity/v3/users/dc698a4b6e204b7fbe196c76f219971a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:47.241393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:47.241883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:47.242306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:47.274334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:47.277187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:47.359713 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2400/4745] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:47.370805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:47.372335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:47.437776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b6898-dbd6-46de-babb-9add0a57ddb0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:47.441535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:47.442211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:47.442595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:47.442942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:47.452578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:47.454027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:47.463781 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2346/4746] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:09:47.753501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:47.755004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:47.768025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83896cce-3e42-4b88-a7cc-848ddc4e042f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:47.773044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:47.773793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:47.774138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:47.774683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] PATH_INFO: `/v3/projects/5bb36de73f044b7e959bf6ec10c56b24` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:47.776946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb36de73f044b7e959bf6ec10c56b24)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:47.779059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:47.833312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb36de73f044b7e959bf6ec10c56b24', 'request_id': 'req-e4c81df6-9ca6-4353-99cc-ce85fee090e5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.839797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb36de73f044b7e959bf6ec10c56b24', 'request_id': 'req-e4c81df6-9ca6-4353-99cc-ce85fee090e5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.846428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb36de73f044b7e959bf6ec10c56b24', 'request_id': 'req-e4c81df6-9ca6-4353-99cc-ce85fee090e5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.849021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:47.863632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] The token cache is being invalidate because project 5bb36de73f044b7e959bf6ec10c56b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:47.863981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.864637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:47.867641 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2401/4747] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:47 2022] DELETE /identity/v3/projects/5bb36de73f044b7e959bf6ec10c56b24 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:47.870710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:47.880846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8546b-c927-49db-8695-c871d59a90ad service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:47.963719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:47.964352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:47.964632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:47.964904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] PATH_INFO: `/v3/users/daf0a04aaa714977abad924150223a99` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:47.966038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf0a04aaa714977abad924150223a99)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:47.968020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:47.981270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:47.983224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:48.028392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf0a04aaa714977abad924150223a99'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.029160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf0a04aaa714977abad924150223a99'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.040028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf0a04aaa714977abad924150223a99'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.053680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c81df6-9ca6-4353-99cc-ce85fee090e5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:48.055536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf0a04aaa714977abad924150223a99'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.056604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf0a04aaa714977abad924150223a99'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.058216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:48.058852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.059127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.059393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.061201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:48.073246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:48.081577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2347/4748] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:47 2022] DELETE /identity/v3/users/daf0a04aaa714977abad924150223a99 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:48.169779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:48.171319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41565a-a9e6-4ded-ac1c-8e8744b0fcda service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:48.186032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2402/4749] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:48.409931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:48.411202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:48.507525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc1bfc-ba4b-4b61-b0fb-adaccc3fe0dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:48.515853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:48.517277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.517564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.517836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] PATH_INFO: `/v3/projects/dc93695013934bf8b244aaf9dee7bc06` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.519048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc93695013934bf8b244aaf9dee7bc06)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:48.521019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:48.575051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc93695013934bf8b244aaf9dee7bc06', 'request_id': 'req-c9bd9b08-2c26-4db9-8519-caced01588cd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.581095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc93695013934bf8b244aaf9dee7bc06', 'request_id': 'req-c9bd9b08-2c26-4db9-8519-caced01588cd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.590683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc93695013934bf8b244aaf9dee7bc06', 'request_id': 'req-c9bd9b08-2c26-4db9-8519-caced01588cd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.602272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] The token cache is being invalidate because project dc93695013934bf8b244aaf9dee7bc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:48.602834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.603548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c9bd9b08-2c26-4db9-8519-caced01588cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:48.605149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2348/4750] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:48 2022] DELETE /identity/v3/projects/dc93695013934bf8b244aaf9dee7bc06 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:48.610233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.610662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.611097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.643134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:48.645294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:48.741517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2403/4751] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:48.743653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.743947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.744767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.772831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:48.774218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:48.787494 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2349/4752] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:48.791728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:48.802432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8433-8e76-4c63-8e09-f30b8a688192 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:48.808576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:48.809199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.809638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.810068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.810903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:48.811383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:48.813560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:48.835294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2404/4753] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:48.837043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:48.848808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a484f-e0e6-4702-be34-ce49c9629527 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:48.855759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:48.856651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:48.857148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:48.857567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:48.858555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:48.861378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:48.915538 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2350/4754] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:48 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:48.920688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:48.934451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9513835-bb15-4fcf-88e6-a4e783b1fdaa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.006481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:49.101813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.102664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.103099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.104138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.105053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.108480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.167930 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2405/4755] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:48 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:49.169182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.185256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cd4e7-7bd3-4070-8998-98977797239e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.190470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.191101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.191378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.192003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.192305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.194120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.212156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2351/4756] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:49.215090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.229894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d6e42-f81c-42f3-9af9-da614edb465d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.237215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.237835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.238112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.238807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.239263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.240913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221336c-518d-454c-b648-81ea493075e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.263226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2406/4757] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:49.264381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.278640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a94f8-38d8-4ec8-ba1f-c41b3ea6027b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.284399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.286259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.286591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.286870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] PATH_INFO: `/v3/projects/4e4535097ac0437e902327afe58c9af6/users/e488f75959f8416097cae4bfd8b4043f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.308614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4535097ac0437e902327afe58c9af6, user_id=e488f75959f8416097cae4bfd8b4043f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.312382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.344574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2352/4758] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:49 2022] PUT /identity/v3/projects/4e4535097ac0437e902327afe58c9af6/users/e488f75959f8416097cae4bfd8b4043f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:49.346293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.347120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.347420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.426949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] MFA Rules not processed for user `e488f75959f8416097cae4bfd8b4043f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:49.428816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:49.533148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2407/4759] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:49.537553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.570594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:49.617329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156c76d-23be-4ee5-bb4e-8d4c03790eea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.624569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.625315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.625693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.626185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.627097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.644728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.680326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2353/4760] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:49 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:49.684069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.698467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fbfa1-24b6-4045-8e4a-92ed38863b74 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.777974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.779106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.779388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.779826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.780621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.783484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.824999 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2408/4761] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:49 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:49.826335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.843981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e5ceb-1a35-4291-992f-5e22750e88f2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.854521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.855567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.855985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.856264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.856916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.858780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.894255 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2354/4762] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:49.897929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:49.927690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d5e2-663e-4569-86ae-9663e1064b5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:49.942206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:49.945308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:49.945630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:49.945910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] PATH_INFO: `/v3/projects/82020e51b087498c967977a03bf963d8/users/e05739db2ca04d239068283379cc4263/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:49.985848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82020e51b087498c967977a03bf963d8, user_id=e05739db2ca04d239068283379cc4263, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:49.989231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:49.993767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:50.033469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.034781 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2409/4763] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:49 2022] PUT /identity/v3/projects/82020e51b087498c967977a03bf963d8/users/e05739db2ca04d239068283379cc4263/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:50.048826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0c29e-bd61-46fd-bce3-e2ef643661c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.207169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.207914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.208228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.212768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.213659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.222944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0b923-bcca-4ebe-8986-d1bfc73cc6c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.245749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2355/4764] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:50.249960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.264916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3154c-68c5-4378-8f54-62691a162f21 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.278775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.279358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.280921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.281456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] PATH_INFO: `/v3/projects/82020e51b087498c967977a03bf963d8/users/e05739db2ca04d239068283379cc4263/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.287018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82020e51b087498c967977a03bf963d8, user_id=e05739db2ca04d239068283379cc4263, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.292934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.311469 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2410/4765] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:50 2022] PUT /identity/v3/projects/82020e51b087498c967977a03bf963d8/users/e05739db2ca04d239068283379cc4263/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:50.323094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.323381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.323649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.373668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] MFA Rules not processed for user `e05739db2ca04d239068283379cc4263`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:50.375321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:50.526777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2356/4766] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:50.534738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.551503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362c628-c7c0-4d95-829a-296d0d6dc390 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.556581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.558025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.558318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.558842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.559685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.560726 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:50.560726 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:09:50.560726 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:09:50.561626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.597904 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2411/4767] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:50 2022] POST /identity/v3/regions => generated 223 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:50.602441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.619856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f85028-2115-4932-bd28-49006b09b334 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.648512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.651222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.651767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.652242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.653158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.654412 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:50.654412 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:09:50.654412 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:09:50.655826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.671193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:50.688568 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2357/4768] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:50 2022] POST /identity/v3/regions => generated 223 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:50.693587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.707687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7d18d-604f-4ba3-8f7c-de383a67a9d6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.729908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.730484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.730846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.731127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/17f7ee3b-c670-41d3-b89d-9241de4d0d93` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.731782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:create_region(region_id=17f7ee3b-c670-41d3-b89d-9241de4d0d93)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.733664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.786433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2412/4769] 10.222.0.52 () {60 vars in 1259 bytes} [Sat Sep 24 21:09:50 2022] PUT /identity/v3/regions/17f7ee3b-c670-41d3-b89d-9241de4d0d93 => generated 231 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:50.788156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.799951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7e39-69ea-46dc-850b-f15938b5fac1 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.821311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.823449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.824273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.824273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/17f7ee3b-c670-41d3-b89d-9241de4d0d93` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.824819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=17f7ee3b-c670-41d3-b89d-9241de4d0d93)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.826914 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:50.826914 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:09:50.826914 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:09:50.828439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.865532 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2358/4770] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:09:50 2022] DELETE /identity/v3/regions/17f7ee3b-c670-41d3-b89d-9241de4d0d93 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:50.867058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.883419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936be5ef-1427-4fca-a512-42e29f7eff7c tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.901788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.903517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.905527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.906118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.907264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.910078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:50.942590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:50.944435 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2413/4771] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:09:50 2022] POST /identity/v3/regions => generated 254 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:50.959697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8e8f-c72b-4bdd-b1a9-181393698007 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:50.982117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:50.982717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:50.983212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:50.983726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:50.984784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:update_region(region_id=815e17dcd8d74b75b2968c8ca4dd2a08)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:50.986034 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:50.986034 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:09:50.986034 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:09:50.987535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.017955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2359/4772] 10.222.0.52 () {60 vars in 1250 bytes} [Sat Sep 24 21:09:50 2022] PATCH /identity/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08 => generated 253 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:51.024382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.055904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f464-b7c1-459e-8304-67a110e03694 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.083627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.085172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.085798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.086856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.087817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:get_region(region_id=815e17dcd8d74b75b2968c8ca4dd2a08)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.090754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.117292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2414/4773] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:09:51 2022] GET /identity/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08 => generated 253 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:51.119002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.141881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8276fc-2fff-4aac-8bf4-278f4a89b506 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.152702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.153941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.153941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.153941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.154474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=815e17dcd8d74b75b2968c8ca4dd2a08)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.156140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.213709 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2360/4774] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:51.215494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.232462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4940e8-4d7d-4308-bb57-e06978d62dd6 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.270949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.271564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.271843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.272118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.272709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.273779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.295703 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2415/4775] 10.222.0.52 () {58 vars in 1128 bytes} [Sat Sep 24 21:09:51 2022] GET /identity/v3/regions => generated 668 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:51.296977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.312357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e1a59-9f09-45f5-ab0c-d7ece6d6a6fd tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.328084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.328738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.329074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.329357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.329949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=815e17dcd8d74b75b2968c8ca4dd2a08)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.331617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Could not find region: 815e17dcd8d74b75b2968c8ca4dd2a08.: keystone.exception.RegionNotFound: Could not find region: 815e17dcd8d74b75b2968c8ca4dd2a08. Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 815e17dcd8d74b75b2968c8ca4dd2a08. Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:09:51.358483 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: 815e17dcd8d74b75b2968c8ca4dd2a08. Sep 24 21:09:51.359592 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:09:51.364693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2361/4776] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/regions/815e17dcd8d74b75b2968c8ca4dd2a08 => generated 112 bytes in 63 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:09:51.369721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.380811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b6502-a342-41e9-8bc9-cb6f058dd255 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.386909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.387949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.388449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.388949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.390003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.391152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.401555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2416/4777] 10.222.0.52 () {58 vars in 1128 bytes} [Sat Sep 24 21:09:51 2022] GET /identity/v3/regions => generated 668 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:51.403214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.417213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44569c-8edc-4321-ad2e-5a189e04fcc8 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.423085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.424026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.424407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.425000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.425827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.428082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.460572 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2362/4778] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:09:51 2022] POST /identity/v3/regions => generated 254 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:51.462336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.477824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924e0d4-58b6-48db-86cf-386a80be6c6b tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.499875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.500454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.500739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.501048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.501740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Adding query filter params (parent_region_id=10aa77daab8d480f801eb2c607fb2c3e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:51.501960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.503829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.513499 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2417/4779] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:09:51 2022] GET /identity/v3/regions?parent_region_id=10aa77daab8d480f801eb2c607fb2c3e => generated 400 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:51.514961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.534750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00229113-4670-4fae-a200-3d107e60ff98 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.541703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.543376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.543376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.543600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/5d318cfb67e44cde8c51e800f42a6b30` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.545002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=5d318cfb67e44cde8c51e800f42a6b30)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.547472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.571799 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2363/4780] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/regions/5d318cfb67e44cde8c51e800f42a6b30 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:51.576623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.592665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007fed6-b149-465f-9290-b8db3d00ccf9 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.643842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.648809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.651430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.652178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/b057ac21ed0e4873a91a2c00a12e08b7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.652981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=b057ac21ed0e4873a91a2c00a12e08b7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.662412 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:09:51.662412 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:09:51.662412 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:09:51.663740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.707149 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2418/4781] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/regions/b057ac21ed0e4873a91a2c00a12e08b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:51.711341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.744915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e7cb9-ed13-490f-96ec-cea3119d456f tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:51.791231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05739db2ca04d239068283379cc4263', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82020e51b087498c967977a03bf963d8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:51.791879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:51.792192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:51.792468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] PATH_INFO: `/v3/regions/10aa77daab8d480f801eb2c607fb2c3e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:51.793051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorizing `identity:delete_region(region_id=10aa77daab8d480f801eb2c607fb2c3e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:51.794702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a4d69-04bd-4167-b486-8487a45c01da tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:51.832426 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2364/4782] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/regions/10aa77daab8d480f801eb2c607fb2c3e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:51.836296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:51.981530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:52.079903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f01d-237e-4b37-9295-02c2d55a9002 tempest-RegionsTestJSON-779456033 tempest-RegionsTestJSON-779456033-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:52.110009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:52.110713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:52.110999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:52.111273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] PATH_INFO: `/v3/users/e488f75959f8416097cae4bfd8b4043f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:52.112460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e488f75959f8416097cae4bfd8b4043f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:52.114664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:52.192335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e488f75959f8416097cae4bfd8b4043f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:52.193936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e488f75959f8416097cae4bfd8b4043f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:52.209169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e488f75959f8416097cae4bfd8b4043f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:52.224091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e488f75959f8416097cae4bfd8b4043f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:52.224863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e488f75959f8416097cae4bfd8b4043f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:52.241891 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2419/4783] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:51 2022] DELETE /identity/v3/users/e488f75959f8416097cae4bfd8b4043f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:52.247066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:52.249022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:52.249727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:52.275044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:52.278885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:52.350408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:52.418263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2365/4784] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:52.550991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:52.554242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:52.737332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778ee87-6561-4b8c-823d-23592b4cef1e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:52.750896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:52.753182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:52.753182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:52.753182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:52.771850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:52.776058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:52.801250 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2420/4785] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:53.600436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:53.603950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:53.626634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76649e7f-53d5-47e7-8781-c29448248303 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:53.645114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:53.646033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:53.647597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:53.648444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:53.651657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:53.772016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:53.777236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:53.820988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2366/4786] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:54.196285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:54.199321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:54.223590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d84379-35b1-410c-8255-b879ffb0e229 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:54.231936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:54.232834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:54.233452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:54.233981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] PATH_INFO: `/v3/projects/4e4535097ac0437e902327afe58c9af6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:54.237010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4535097ac0437e902327afe58c9af6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:54.239372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:54.324309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:54.350394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4535097ac0437e902327afe58c9af6', 'request_id': 'req-6d36b85b-05da-4c6d-aef4-0aa250a706ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.360724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4535097ac0437e902327afe58c9af6', 'request_id': 'req-6d36b85b-05da-4c6d-aef4-0aa250a706ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.389105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4535097ac0437e902327afe58c9af6', 'request_id': 'req-6d36b85b-05da-4c6d-aef4-0aa250a706ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.409459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] The token cache is being invalidate because project 4e4535097ac0437e902327afe58c9af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:54.409929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.410943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.415672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2421/4787] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:54 2022] DELETE /identity/v3/projects/4e4535097ac0437e902327afe58c9af6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:54.416847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:54.439006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41757267-4e74-474b-944b-4310d46e1bc1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:54.584925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:54.587058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:54.590176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:54.591031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] PATH_INFO: `/v3/users/e05739db2ca04d239068283379cc4263` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:54.595285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05739db2ca04d239068283379cc4263)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:54.599414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:54.746236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05739db2ca04d239068283379cc4263'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.746961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05739db2ca04d239068283379cc4263'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.757242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05739db2ca04d239068283379cc4263'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.779181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05739db2ca04d239068283379cc4263'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.780022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05739db2ca04d239068283379cc4263'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:54.799186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:54.824190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2367/4788] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:54 2022] DELETE /identity/v3/users/e05739db2ca04d239068283379cc4263 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:55.939412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:55.940982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:55.956656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36b85b-05da-4c6d-aef4-0aa250a706ea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:56.026689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:56.026689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.026689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.026689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] PATH_INFO: `/v3/users/fdbd9a2294414c0e932ef46d11f8e4b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.027340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbd9a2294414c0e932ef46d11f8e4b6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:56.029230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:56.085416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbd9a2294414c0e932ef46d11f8e4b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.086089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbd9a2294414c0e932ef46d11f8e4b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.095458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbd9a2294414c0e932ef46d11f8e4b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.114241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbd9a2294414c0e932ef46d11f8e4b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.115176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-670edd03-a501-47ae-865f-a5a26051b5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbd9a2294414c0e932ef46d11f8e4b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.133908 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2422/4789] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:09:55 2022] DELETE /identity/v3/users/fdbd9a2294414c0e932ef46d11f8e4b6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:56.283854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:56.285345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:56.303540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304e3d2-5186-47f3-a594-283e0fa19ef2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:56.428430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:56.430062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.433162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.433934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] PATH_INFO: `/v3/projects/82020e51b087498c967977a03bf963d8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.436853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82020e51b087498c967977a03bf963d8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:56.440145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:56.512388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82020e51b087498c967977a03bf963d8', 'request_id': 'req-0d63a294-22b3-438b-98c1-2a3b92693411'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.519229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82020e51b087498c967977a03bf963d8', 'request_id': 'req-0d63a294-22b3-438b-98c1-2a3b92693411'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.528916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82020e51b087498c967977a03bf963d8', 'request_id': 'req-0d63a294-22b3-438b-98c1-2a3b92693411'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.544739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] The token cache is being invalidate because project 82020e51b087498c967977a03bf963d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:56.545140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.545835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.548280 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2368/4790] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:56 2022] DELETE /identity/v3/projects/82020e51b087498c967977a03bf963d8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:56.556227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277f059-2779-4dbe-8de9-534e78744fdc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.557202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277f059-2779-4dbe-8de9-534e78744fdc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.558101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277f059-2779-4dbe-8de9-534e78744fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.582191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8277f059-2779-4dbe-8de9-534e78744fdc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:56.583721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8277f059-2779-4dbe-8de9-534e78744fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:56.599496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:56.604916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:56.662734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:56.715221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63a294-22b3-438b-98c1-2a3b92693411 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:56.721525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:56.722286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.722714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.723081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] PATH_INFO: `/v3/projects/8428ee98cbdc457192d9889bf10d1036` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.724458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8428ee98cbdc457192d9889bf10d1036)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:56.727178 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2423/4791] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:56.735622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:56.739427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.741618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.742180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.789261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:56.790716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:56.804284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2424/4792] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:56.805395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:56.808525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:56.821239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf467b7-74e1-466a-a8a2-e43d0cdc1554 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:56.840805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:56.841513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.841755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.842034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.842847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:56.843113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:56.844773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:56.861221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8428ee98cbdc457192d9889bf10d1036', 'request_id': 'req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.866041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2425/4793] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:56.866993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:56.875530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8428ee98cbdc457192d9889bf10d1036', 'request_id': 'req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.879869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22553a8-abd5-4e4b-84fa-5c527a373521 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:56.918150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8428ee98cbdc457192d9889bf10d1036', 'request_id': 'req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.944904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] The token cache is being invalidate because project 8428ee98cbdc457192d9889bf10d1036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:09:56.945201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.945901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e4eb2e0-db92-422b-b00c-ba01ed122bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:09:56.948577 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2369/4794] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:09:56 2022] DELETE /identity/v3/projects/8428ee98cbdc457192d9889bf10d1036 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:56.954238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a72ef-bcf8-4c79-aa73-0607e8a296d8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:56.954546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a72ef-bcf8-4c79-aa73-0607e8a296d8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:56.954901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a72ef-bcf8-4c79-aa73-0607e8a296d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:56.993392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be5a72ef-bcf8-4c79-aa73-0607e8a296d8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:56.994844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be5a72ef-bcf8-4c79-aa73-0607e8a296d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:57.002957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.003573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.004205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.004635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.005406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.009238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.037856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2370/4795] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:57.048079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.048616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.049049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.069977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2426/4796] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:56 2022] POST /identity/v3/projects => generated 367 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:57.071170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.073530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:57.080197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:57.081416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:57.202386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da64973-2b16-4431-83c2-5cf0891219ec admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.211239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.212364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.213418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.213795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.214548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.217006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.226875 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2371/4797] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:57.227961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.247206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c797d-4685-4516-9f3d-a4854b52a25f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.253214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.253989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.254292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.255343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.255980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:09:57.256345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.258459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.258617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2427/4798] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:57.260084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.285170 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2372/4799] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:09:57.286189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.292685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c94b74-3035-42b0-b6ba-fc3d58f57f3f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.299879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d10d-4a63-4f1a-bfcd-dbc27d56773f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.305250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.305541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.306082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.306198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.306457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.306764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.306931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.307309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.307423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.308906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.309044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.312015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.333652 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2428/4800] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:57.338945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.351343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5fa30-d1e2-431b-b611-9972b3bb0e31 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.366787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.368371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.368771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.369860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.371009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.374525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.377629 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2373/4801] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:57.380385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.404774 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2429/4802] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:57.410103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.435859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1cbea-0d66-4331-b1a4-e424bb19de13 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.443031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:57.545030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ca370-6e27-423d-85c4-7f4905ac7d76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.553006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.554528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.555002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.555425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] PATH_INFO: `/v3/projects/c53745083f98428eba664cb022aff9ef/users/7ca1fa6f8d074fcf895eac45141f6ac9/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.557561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.558307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.558676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.558963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.559672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.563598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.583797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53745083f98428eba664cb022aff9ef, user_id=7ca1fa6f8d074fcf895eac45141f6ac9, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.588576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cac680-7aa4-4df4-bbef-226d70197838 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.597127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:57.623175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2374/4803] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:57.623949 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2430/4804] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:57 2022] PUT /identity/v3/projects/c53745083f98428eba664cb022aff9ef/users/7ca1fa6f8d074fcf895eac45141f6ac9/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:57.624582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.637011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10a463-b79c-4394-81d7-6274b2e47986 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.640556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.641225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.641751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.691572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] MFA Rules not processed for user `7ca1fa6f8d074fcf895eac45141f6ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:57.696315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:57.828379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.829025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.829297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.829579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.830903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.834210 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2431/4805] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:09:57.835058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.839006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.866062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadab34-e698-4219-a15a-ad97132d8890 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.867262 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2375/4806] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:57.868195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.882924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.883570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.883852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.884127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.888934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.892335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.898629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5669c5-6566-4844-bdd0-82a79393da3e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:57.928664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:57.929325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:57.929613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:57.929891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:57.930559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:57.932849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:57.953991 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2376/4807] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:57.964056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.983026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2432/4808] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:57.984381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:57.996952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b7d9-4837-435f-bed0-b5027168e73f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.104324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.105551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.105836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.106115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] PATH_INFO: `/v3/projects/7137f08b68d046bc9017aa466b7da5a9/users/d9c24f9c3b0843fe831eeb8e43b0328a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.125506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7137f08b68d046bc9017aa466b7da5a9, user_id=d9c24f9c3b0843fe831eeb8e43b0328a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.128991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd34365-a56a-4207-82ac-a0b0a1455fa1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.132072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e3804-5416-4883-9216-e1080dff659c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.149185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.149469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.149723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.149994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.150845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.152823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2377/4809] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:57 2022] PUT /identity/v3/projects/7137f08b68d046bc9017aa466b7da5a9/users/d9c24f9c3b0843fe831eeb8e43b0328a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:58.159040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.160436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.161051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.161521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.176579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:58.191753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:58.193402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:58.198879 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2433/4810] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:57 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:58.203442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:58.215752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd98f9-f877-426c-8f83-d6512e353d5f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.280760 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2378/4811] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:58.298895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.300488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.300488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.300679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.301274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.301644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:58.303054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.304644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:58.316473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:58.323066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2434/4812] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:58.327466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:58.352537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422049db-b8fc-46cd-ba37-549a226cea12 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.369166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.371143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.371420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.371695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] PATH_INFO: `/v3/projects/68b0301b068a4b149ef23157daf674cf/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.399984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfeeac-5a40-4b89-a6ab-bd1f904a476b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.409976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.413729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.414576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.414576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.418371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b0301b068a4b149ef23157daf674cf, user_id=e0027ae93d7843d1954ee7ba2b201aed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.421356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.427678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.429146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.448115 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2379/4813] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:09:58.449529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2435/4814] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:58 2022] PUT /identity/v3/projects/68b0301b068a4b149ef23157daf674cf/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:58.453727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:58.465402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adbcd3-44bc-480a-ab7f-0743d6223e19 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.611062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.611874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.612261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.612552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.613383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.615334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc464fe-e219-4d55-9c89-e9c1d6d6030d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.636519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2380/4815] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:58.637009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:58.652770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2907-865d-4505-8bde-a6492308a5de admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:58.665444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:58.667466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.667981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.668505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] PATH_INFO: `/v3/projects/68b0301b068a4b149ef23157daf674cf/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.671632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b0301b068a4b149ef23157daf674cf, user_id=e0027ae93d7843d1954ee7ba2b201aed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:58.676395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:58.755616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:58.755858 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2436/4816] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:58 2022] PUT /identity/v3/projects/68b0301b068a4b149ef23157daf674cf/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:09:58.756402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:58.756886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:58.825841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] MFA Rules not processed for user `e0027ae93d7843d1954ee7ba2b201aed`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:58.827525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:58.928204 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2381/4817] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:58.936015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.068495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3eb0dc-2566-4194-84f7-d03acb2ec06a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.074709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.075413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.075771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.076154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.076919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.079354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.149429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.150816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2437/4818] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:09:58 2022] POST /identity/v3/projects => generated 369 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:59.169992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac5756-f524-44dd-956c-e5510aaf176e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.219155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:59.281606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.282235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.282543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.282824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.283465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.285527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.318753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2382/4819] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:09:59 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:09:59.322191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.342348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeaf2b-f1bb-4d64-aaa4-8872a557f7a1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.350159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.351216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.352544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.353056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.354716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.356635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.374609 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2438/4820] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:09:59.379159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.393605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dda5d8-26fe-4cfd-814d-3a27a1a51fb2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.404586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.405202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.405481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.405750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.406683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.408332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1f94d-495d-4082-9bc0-c7462d16435f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.440220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2383/4821] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:09:59.443696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.459604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd2371-5be4-4660-94db-da61a0865635 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.465413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.466120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.466400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.466723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] PATH_INFO: `/v3/projects/291b5747bb35450aaeedfde7912efefd/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.492632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291b5747bb35450aaeedfde7912efefd, user_id=5decbb0b75394a6c8678d7f6bd4092cc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.496778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.514775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:09:59.543054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2439/4822] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:09:59 2022] PUT /identity/v3/projects/291b5747bb35450aaeedfde7912efefd/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:09:59.555238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.555686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.556078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.595439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] MFA Rules not processed for user `5decbb0b75394a6c8678d7f6bd4092cc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:09:59.597382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:09:59.727023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2384/4823] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:09:59.728530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.823019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207e5b-ff56-462b-860d-17d90e63a2a1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.829042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.829762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.830120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.830655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.831400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.833467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:09:59.867383 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2440/4824] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:09:59 2022] POST /identity/v3/domains => generated 277 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:09:59.868603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:09:59.881808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3623a-a7ce-4575-aa9d-0beefe027092 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:09:59.981269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:09:59.981928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:09:59.982798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:09:59.983094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:09:59.985563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:09:59.987451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.017511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2385/4825] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:09:59 2022] POST /identity/v3/roles => generated 240 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:00.021748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.037557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f013fa2-6ccc-49a3-ba41-18291a79520d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.046342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.047072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.047353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.047629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/cfcab4ef51484975bb132b9c9762c024/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/db9d4b23c1f244bd93b2266783a85d86` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.067915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=cfcab4ef51484975bb132b9c9762c024, user_id=5decbb0b75394a6c8678d7f6bd4092cc, role_id=db9d4b23c1f244bd93b2266783a85d86)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.070972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.097709 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2441/4826] 10.222.0.52 () {60 vars in 1480 bytes} [Sat Sep 24 21:10:00 2022] PUT /identity/v3/domains/cfcab4ef51484975bb132b9c9762c024/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/db9d4b23c1f244bd93b2266783a85d86 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:00.099032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.119877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea3dcb-0a74-4d9b-a70b-5adc9519a61b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.221978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:00.318048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.318707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.318992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.319516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.319924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.321584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.378187 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2386/4827] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:10:00 2022] POST /identity/v3/domains => generated 276 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:00.379376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.396103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756ba54-f79b-4645-9710-ffa99136f09e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.512738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.522878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.523233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.523678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.525509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.527982 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:00.527982 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:00.527982 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:00.531437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.571125 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2442/4828] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:00 2022] POST /identity/v3/roles => generated 239 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:00.576042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.590671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c11ccb-1462-4a2f-aba3-46aea70cc831 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.602179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.602835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.603704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.604006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.604607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.605632 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:00.605632 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:00.605632 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:00.611086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.688822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2387/4829] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:10:00 2022] POST /identity/v3/groups => generated 327 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:00.693384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.717005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d83e0e-c116-4fc2-8a20-ddfd48eec078 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.732590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.733878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.734406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.734852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659/users/5decbb0b75394a6c8678d7f6bd4092cc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.752780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=b745ace4c3cc4cbebbf0e15e3244d659, user_id=5decbb0b75394a6c8678d7f6bd4092cc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.755207 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:00.755207 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:00.755207 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:00.757139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.844349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2443/4830] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:10:00 2022] PUT /identity/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659/users/5decbb0b75394a6c8678d7f6bd4092cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:00.848707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.871397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d83f-444c-4e90-a1d2-feeaaba3a144 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:00.940820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:00.941839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:00.942226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:00.942683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/182a27229a8f49e8bd8d8ab2e082a849/groups/b745ace4c3cc4cbebbf0e15e3244d659/roles/d5650869c95f47c8baad0df98923b0d6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:00.945992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=182a27229a8f49e8bd8d8ab2e082a849, group_id=b745ace4c3cc4cbebbf0e15e3244d659, role_id=d5650869c95f47c8baad0df98923b0d6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:00.949685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:00.980705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:00.981103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2388/4831] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:00 2022] PUT /identity/v3/domains/182a27229a8f49e8bd8d8ab2e082a849/groups/b745ace4c3cc4cbebbf0e15e3244d659/roles/d5650869c95f47c8baad0df98923b0d6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:01.005392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257b383-345e-40a1-b3cb-e47df462abbb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:01.095101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:01.169328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:01.170664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.173509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.174841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.175677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:01.178389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:01.248697 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2444/4832] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:10:00 2022] POST /identity/v3/domains => generated 275 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:01.254563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:01.289806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9d47-8e20-4135-ad13-c1fac7e0dc87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:01.489239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:01.490139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.490545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.490934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.491733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:01.494852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:01.530465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2389/4833] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:01 2022] POST /identity/v3/roles => generated 240 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:01.535160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:01.561260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787358c-7e81-49e4-a335-b477fde6bce0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:01.570877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:01.572488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.572965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.573448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/d79f0f43165c4528a366c2a5d7efcbea/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/4b9ee4324e5642e3aff2bc5afb378c5d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.607452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d79f0f43165c4528a366c2a5d7efcbea, user_id=5decbb0b75394a6c8678d7f6bd4092cc, role_id=4b9ee4324e5642e3aff2bc5afb378c5d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:01.613433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:01.640131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2445/4834] 10.222.0.52 () {60 vars in 1480 bytes} [Sat Sep 24 21:10:01 2022] PUT /identity/v3/domains/d79f0f43165c4528a366c2a5d7efcbea/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/4b9ee4324e5642e3aff2bc5afb378c5d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:01.644642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:01.675479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0350-54a5-4a03-b1b3-3189a2f2b9af tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:01.729614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:01.780916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:01.782108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.782522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.782888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.783684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:01.785500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182408cf-0356-480c-8b5a-9b4083ce3626 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:01.826171 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2390/4835] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:10:01 2022] POST /identity/v3/domains => generated 276 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:01.833626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:01.855737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259828-7440-4bcf-9274-e687fa425e79 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:01.862000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.862286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.862696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.898603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:01.898603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:01.972981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:01.974432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:01.974866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:01.975233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:01.976029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:01.978403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.017288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2446/4836] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:01 2022] POST /identity/v3/roles => generated 239 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:02.021318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.038545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19650e20-dacb-45b5-a7ef-1a4f4c69c36c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:02.042256 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2391/4837] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:02.045489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:02.046321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:02.046798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:02.047239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:02.048028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:02.049635 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:02.049635 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:02.049635 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:02.051714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.123711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.126526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:02.136699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2447/4838] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:10:02 2022] POST /identity/v3/groups => generated 326 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:02.140789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.162457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d314c0f-32ef-4fbc-ac18-3ccf401fce8f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:02.172010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:02.172919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:02.173355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:02.174715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/10bb577f5c9641cbb128e67946b13b7d/users/5decbb0b75394a6c8678d7f6bd4092cc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:02.183649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=10bb577f5c9641cbb128e67946b13b7d, user_id=5decbb0b75394a6c8678d7f6bd4092cc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:02.185839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.226486 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2448/4839] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:10:02 2022] PUT /identity/v3/groups/10bb577f5c9641cbb128e67946b13b7d/users/5decbb0b75394a6c8678d7f6bd4092cc => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:02.230808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.335696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3590784-c653-43b9-ba45-e37f44093621 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:02.417394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:02.424252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd39c7-7d6d-49a7-9773-3a949769863c tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:02.429857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:02.430902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:02.432560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:02.432829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/c81778c51b1a4f64b5fb333b4561d84e/groups/10bb577f5c9641cbb128e67946b13b7d/roles/4a5c8744071f46cbb7b237ec0cadd819` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:02.437379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=c81778c51b1a4f64b5fb333b4561d84e, group_id=10bb577f5c9641cbb128e67946b13b7d, role_id=4a5c8744071f46cbb7b237ec0cadd819)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:02.440741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.466837 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2449/4840] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:02 2022] PUT /identity/v3/domains/c81778c51b1a4f64b5fb333b4561d84e/groups/10bb577f5c9641cbb128e67946b13b7d/roles/4a5c8744071f46cbb7b237ec0cadd819 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:02.467141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:02.471693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.509126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:02.514165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:02.515420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:02.515967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:02.731646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:02.733607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.755934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2392/4841] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:10:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:10:02.762393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878b1b3-3b1a-4181-9611-0a9df41c417f tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:02.768619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5decbb0b75394a6c8678d7f6bd4092cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '291b5747bb35450aaeedfde7912efefd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:02.769049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:02.769491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:02.770606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] PATH_INFO: `/v3/auth/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:02.771269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:02.772669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:02.961802 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2450/4842] 10.222.0.52 () {58 vars in 1143 bytes} [Sat Sep 24 21:10:02 2022] GET /identity/v3/auth/domains => generated 1172 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:10:02.963698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:02.982361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b7281-3ead-48d6-8d20-5b1549abcf66 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:03.128446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:03.129228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:03.129513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:03.129792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/c81778c51b1a4f64b5fb333b4561d84e/groups/10bb577f5c9641cbb128e67946b13b7d/roles/4a5c8744071f46cbb7b237ec0cadd819` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:03.136016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=c81778c51b1a4f64b5fb333b4561d84e, group_id=10bb577f5c9641cbb128e67946b13b7d, role_id=4a5c8744071f46cbb7b237ec0cadd819)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:03.137150 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:03.137150 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:03.137150 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:03.145014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:03.165779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 4a5c8744071f46cbb7b237ec0cadd819 was removed from group 10bb577f5c9641cbb128e67946b13b7d on domain c81778c51b1a4f64b5fb333b4561d84e. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:03.166081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:03.166925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:03.197936 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2393/4843] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:10:02 2022] DELETE /identity/v3/domains/c81778c51b1a4f64b5fb333b4561d84e/groups/10bb577f5c9641cbb128e67946b13b7d/roles/4a5c8744071f46cbb7b237ec0cadd819 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:03.202223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:03.204025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:03.229797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18aa45-3e23-4edf-a759-360eb1d09caa tempest-TokensV3TestJSON-1023589369 tempest-TokensV3TestJSON-1023589369-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:03.356854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:03.357859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:03.358287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:03.358747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/10bb577f5c9641cbb128e67946b13b7d/users/5decbb0b75394a6c8678d7f6bd4092cc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:03.364808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=10bb577f5c9641cbb128e67946b13b7d, user_id=5decbb0b75394a6c8678d7f6bd4092cc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:03.366882 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:03.366882 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:03.366882 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:03.368990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:03.385123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:03.402302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:03.435479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2451/4844] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:10:03 2022] DELETE /identity/v3/groups/10bb577f5c9641cbb128e67946b13b7d/users/5decbb0b75394a6c8678d7f6bd4092cc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:03.437071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:03.453257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a874f-3f15-451d-a691-92117c1f093e tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:03.610935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:03.611904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:03.612302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:03.612679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/10bb577f5c9641cbb128e67946b13b7d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:03.614865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_group(group_id=10bb577f5c9641cbb128e67946b13b7d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:03.616913 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:03.616913 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:03.616913 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:03.624460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:03.755185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:03.791096 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2394/4845] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:10:03 2022] DELETE /identity/v3/groups/10bb577f5c9641cbb128e67946b13b7d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:03.792185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:03.811530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2976-9a96-4cb2-8265-e504a04a081a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:03.990034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:03.990914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:03.991201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:03.991494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles/4a5c8744071f46cbb7b237ec0cadd819` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:03.993373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4a5c8744071f46cbb7b237ec0cadd819)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:04.000159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:04.077866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 4a5c8744071f46cbb7b237ec0cadd819 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:04.078242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.078959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.081925 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2452/4846] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:03 2022] DELETE /identity/v3/roles/4a5c8744071f46cbb7b237ec0cadd819 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:04.086290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:04.088386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:04.123045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513370d-4227-4cb7-8e25-9ffefd2feaa9 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:04.326264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:04.327090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:04.327420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:04.327699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/c81778c51b1a4f64b5fb333b4561d84e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:04.332136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=c81778c51b1a4f64b5fb333b4561d84e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:04.334024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:04.368709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.370474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.440757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.442573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.448300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2395/4847] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:04 2022] PATCH /identity/v3/domains/c81778c51b1a4f64b5fb333b4561d84e => generated 277 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:04.452636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:04.467684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78d05-ce45-4e85-84e9-2549bba773ea tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:04.689904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:04.690903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:04.691188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:04.692251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/c81778c51b1a4f64b5fb333b4561d84e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:04.721141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=c81778c51b1a4f64b5fb333b4561d84e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:04.725182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:04.803760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.814944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:04.934067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:05.012556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.085380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.118349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.170790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project c81778c51b1a4f64b5fb333b4561d84e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:05.171160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.172602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.198781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.208390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.211003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'c81778c51b1a4f64b5fb333b4561d84e', 'request_id': 'req-ac85a5ac-a828-4f3d-8286-c36223abaeff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.254043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2453/4848] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:10:04 2022] DELETE /identity/v3/domains/c81778c51b1a4f64b5fb333b4561d84e => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:05.258839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:05.261710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:05.307006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d5f2-9909-49cb-8f09-ad8a0088963d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:05.483325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:05.485880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:05.487720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:05.489101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/d79f0f43165c4528a366c2a5d7efcbea/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/4b9ee4324e5642e3aff2bc5afb378c5d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:05.509416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d79f0f43165c4528a366c2a5d7efcbea, user_id=5decbb0b75394a6c8678d7f6bd4092cc, role_id=4b9ee4324e5642e3aff2bc5afb378c5d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:05.513013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:05.536727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 4b9ee4324e5642e3aff2bc5afb378c5d was removed from user 5decbb0b75394a6c8678d7f6bd4092cc on domain d79f0f43165c4528a366c2a5d7efcbea. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:05.539795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.543097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.582434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2396/4849] 10.222.0.52 () {58 vars in 1464 bytes} [Sat Sep 24 21:10:05 2022] DELETE /identity/v3/domains/d79f0f43165c4528a366c2a5d7efcbea/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/4b9ee4324e5642e3aff2bc5afb378c5d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:05.586837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:05.589924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:05.622945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a5ac-a828-4f3d-8286-c36223abaeff tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:05.805869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:05.807271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:05.807690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:05.808068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles/4b9ee4324e5642e3aff2bc5afb378c5d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:05.810763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4b9ee4324e5642e3aff2bc5afb378c5d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:05.821737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:05.886999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:05.890712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:05.899398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 4b9ee4324e5642e3aff2bc5afb378c5d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:05.902736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.903931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:05.916137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2454/4850] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:05 2022] DELETE /identity/v3/roles/4b9ee4324e5642e3aff2bc5afb378c5d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:05.917377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:05.921157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:06.032748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:06.107410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6af3-4b9a-4a04-8898-5ef317a3e326 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:06.122963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:06.133093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd570a4-6dc5-413d-9182-3f85d4e54303 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:06.157277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:06.158086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:06.158482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:06.158909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/d79f0f43165c4528a366c2a5d7efcbea` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:06.176329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d79f0f43165c4528a366c2a5d7efcbea)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:06.176329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:06.241808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.246325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.337444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:06.338430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:06.340668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:06.341087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] PATH_INFO: `/v3/users/9eca15a31e064f4199bfc4b04de7c5b2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:06.343512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eca15a31e064f4199bfc4b04de7c5b2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:06.346970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.347895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:06.348351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.356052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2455/4851] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:05 2022] PATCH /identity/v3/domains/d79f0f43165c4528a366c2a5d7efcbea => generated 276 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:06.360885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:06.367485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:06.478125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eca15a31e064f4199bfc4b04de7c5b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.479010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eca15a31e064f4199bfc4b04de7c5b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.500954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eca15a31e064f4199bfc4b04de7c5b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.513291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eca15a31e064f4199bfc4b04de7c5b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.514326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eca15a31e064f4199bfc4b04de7c5b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.591403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2397/4852] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:05 2022] DELETE /identity/v3/users/9eca15a31e064f4199bfc4b04de7c5b2 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:06.612795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc828f-66ff-4a7e-8cbf-089735122271 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:06.767237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:06.768023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:06.768379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:06.768769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/d79f0f43165c4528a366c2a5d7efcbea` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:06.795179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d79f0f43165c4528a366c2a5d7efcbea)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:06.799289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:06.862871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.871515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:06.973700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:06.988839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.003154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.045046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.085154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project d79f0f43165c4528a366c2a5d7efcbea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:07.085475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.090848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.121987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.132161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.132945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd79f0f43165c4528a366c2a5d7efcbea', 'request_id': 'req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.146795 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2456/4853] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:10:06 2022] DELETE /identity/v3/domains/d79f0f43165c4528a366c2a5d7efcbea => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:07.152835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:07.154875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:07.181672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041623d6-7370-41d5-9983-fb720875c902 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:07.437458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:07.438213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:07.439512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:07.439861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/182a27229a8f49e8bd8d8ab2e082a849/groups/b745ace4c3cc4cbebbf0e15e3244d659/roles/d5650869c95f47c8baad0df98923b0d6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:07.454259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=182a27229a8f49e8bd8d8ab2e082a849, group_id=b745ace4c3cc4cbebbf0e15e3244d659, role_id=d5650869c95f47c8baad0df98923b0d6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:07.467398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:07.488869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role d5650869c95f47c8baad0df98923b0d6 was removed from group b745ace4c3cc4cbebbf0e15e3244d659 on domain 182a27229a8f49e8bd8d8ab2e082a849. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:07.488869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.490063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.501099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:07.529184 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2398/4854] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:10:07 2022] DELETE /identity/v3/domains/182a27229a8f49e8bd8d8ab2e082a849/groups/b745ace4c3cc4cbebbf0e15e3244d659/roles/d5650869c95f47c8baad0df98923b0d6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:07.533876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:07.535530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:07.564483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5543a-2cf1-4c74-ba1d-ebc9d5614b87 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:07.748167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:07.748911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:07.749200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:07.749482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659/users/5decbb0b75394a6c8678d7f6bd4092cc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:07.753290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=b745ace4c3cc4cbebbf0e15e3244d659, user_id=5decbb0b75394a6c8678d7f6bd4092cc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:07.766243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:07.792410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:07.825475 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2457/4855] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:10:07 2022] DELETE /identity/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659/users/5decbb0b75394a6c8678d7f6bd4092cc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:07.827328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:07.829157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:07.870968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ae62-740f-4096-ae7a-f8e398cd3820 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:08.006234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:08.007682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.008190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.010202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.012217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_group(group_id=b745ace4c3cc4cbebbf0e15e3244d659)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:08.014647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:08.040311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:08.044153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:08.153708 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2399/4856] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:10:07 2022] DELETE /identity/v3/groups/b745ace4c3cc4cbebbf0e15e3244d659 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:08.170433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:08.197236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3238f7-111f-41be-8c86-a3fa7748ebdb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:08.303042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:08.331611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:08.332432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.332714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.332989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] PATH_INFO: `/v3/projects/d0e5867b34544cadbb354439808248aa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.334802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e5867b34544cadbb354439808248aa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:08.347947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:08.372142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8809fa9-9c84-456b-9b52-4a28a7911401 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:08.386463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:08.390228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.390532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.390818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles/d5650869c95f47c8baad0df98923b0d6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.400598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d5650869c95f47c8baad0df98923b0d6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:08.401878 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:08.401878 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:08.401878 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:08.402832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:08.481357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e5867b34544cadbb354439808248aa', 'request_id': 'req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.495821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e5867b34544cadbb354439808248aa', 'request_id': 'req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.509575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e5867b34544cadbb354439808248aa', 'request_id': 'req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.564874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] The token cache is being invalidate because project d0e5867b34544cadbb354439808248aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:08.565297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.566941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a2e9656-ce91-4b4e-8b72-70faf1d2b00d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.573086 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2458/4857] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:08 2022] DELETE /identity/v3/projects/d0e5867b34544cadbb354439808248aa => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:08.574998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6f3cf-8ad8-4565-b3c1-b2a9c5b7c125 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.575846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6f3cf-8ad8-4565-b3c1-b2a9c5b7c125 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.575846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6f3cf-8ad8-4565-b3c1-b2a9c5b7c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.601703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role d5650869c95f47c8baad0df98923b0d6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:08.605878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.609541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.623527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2400/4858] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:08 2022] DELETE /identity/v3/roles/d5650869c95f47c8baad0df98923b0d6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:08.650693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:08.652708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fdd6f3cf-8ad8-4565-b3c1-b2a9c5b7c125 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:08.653610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:08.664061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fdd6f3cf-8ad8-4565-b3c1-b2a9c5b7c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:08.700547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:08.831083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196dcc4-be0f-431f-aa5d-36712d532378 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:08.858699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2459/4859] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:08.859253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:08.859901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.860192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.860318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.860473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/182a27229a8f49e8bd8d8ab2e082a849` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.860589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.860768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.870741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=182a27229a8f49e8bd8d8ab2e082a849)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:08.872418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:08.910154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:08.911521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:08.940622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2460/4860] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:08.944188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:08.957347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.960327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5a3da-e8f1-4946-b10b-6b6563950ec5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:08.961120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:08.975151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:08.975973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:08.976117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:08.976568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:08.977245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:08.977550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:08.984671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.014041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2461/4861] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:09.019028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.027790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:09.064666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1164-2250-43d4-bb73-53b3edb62506 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.075165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.078785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.112315 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2401/4862] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:08 2022] PATCH /identity/v3/domains/182a27229a8f49e8bd8d8ab2e082a849 => generated 277 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:09.115273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.119309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:09.162474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:09.178638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.179326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.180231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.181051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.182335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.186024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.242542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2462/4863] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:09 2022] POST /identity/v3/projects => generated 371 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:09.246671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.260095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:09.375285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b99363-f391-444b-8c60-fc19831cfa3a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.393202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37b127-aa9a-401e-8ccd-632876cfaf92 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.402001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.402671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.402954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.403227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.403869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.405816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.456483 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2463/4864] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:09 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:09.460231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.485656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083afd0-669f-49c7-80af-6064cdf443c0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.502778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.503410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.503693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.503968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.504614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.506238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.531627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2464/4865] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:09 2022] GET /identity/v3/roles => generated 1651 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:10:09.532807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.568076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f8990-7c87-44e3-b518-3639ed100fdd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.588193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.588763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.588895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.589126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.589389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.589512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/182a27229a8f49e8bd8d8ab2e082a849` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.589692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.589939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.590622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.592249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.612489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=182a27229a8f49e8bd8d8ab2e082a849)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.614395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.622549 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2465/4866] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:09 2022] GET /identity/v3/roles => generated 1651 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:10:09.655459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.655557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.661224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:09.669081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca64ce-aba8-4c14-a34e-6f99870e18d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:09.677734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.678701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:09.679387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.679668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.679943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] PATH_INFO: `/v3/projects/09b303d9b041415aa4b96dbb7dfe4dae/users/dc0a3805b2924155a370e40d79091eec/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.701371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b303d9b041415aa4b96dbb7dfe4dae, user_id=dc0a3805b2924155a370e40d79091eec, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:09.704131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21b1e6-6c5a-43b4-9f47-95debfabf8d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:09.721470 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2466/4867] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:09 2022] PUT /identity/v3/projects/09b303d9b041415aa4b96dbb7dfe4dae/users/dc0a3805b2924155a370e40d79091eec/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:09.734404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:09.734752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:09.735027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:09.769482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.774010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] MFA Rules not processed for user `dc0a3805b2924155a370e40d79091eec`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:09.777293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:09.783503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.804477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.832107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project 182a27229a8f49e8bd8d8ab2e082a849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:09.832730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.835491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.858257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.860520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.870419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '182a27229a8f49e8bd8d8ab2e082a849', 'request_id': 'req-6114c843-adf5-419f-8e38-4bd22f7c1091'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:09.887472 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2402/4868] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:10:09 2022] DELETE /identity/v3/domains/182a27229a8f49e8bd8d8ab2e082a849 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:09.889046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.891692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:09.944791 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2467/4869] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:09.956183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:09.958533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:10.016093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:10.049087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114c843-adf5-419f-8e38-4bd22f7c1091 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:10.070608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:10.071420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:10.071753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:10.072039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/cfcab4ef51484975bb132b9c9762c024/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/db9d4b23c1f244bd93b2266783a85d86` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:10.085333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=cfcab4ef51484975bb132b9c9762c024, user_id=5decbb0b75394a6c8678d7f6bd4092cc, role_id=db9d4b23c1f244bd93b2266783a85d86)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:10.088446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:10.105685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role db9d4b23c1f244bd93b2266783a85d86 was removed from user 5decbb0b75394a6c8678d7f6bd4092cc on domain cfcab4ef51484975bb132b9c9762c024. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:10.106076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.108765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.148474 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2403/4870] 10.222.0.52 () {58 vars in 1464 bytes} [Sat Sep 24 21:10:09 2022] DELETE /identity/v3/domains/cfcab4ef51484975bb132b9c9762c024/users/5decbb0b75394a6c8678d7f6bd4092cc/roles/db9d4b23c1f244bd93b2266783a85d86 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:10.152850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:10.160676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:10.300571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71231e-d730-4867-b970-56503bca6448 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:10.320903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:10.321766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:10.322057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:10.322345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles/db9d4b23c1f244bd93b2266783a85d86` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:10.326961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_role(role_id=db9d4b23c1f244bd93b2266783a85d86)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:10.328591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:10.335687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964506f5-ad71-460f-b848-ba4ece3430aa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:10.340645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:10.436631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role db9d4b23c1f244bd93b2266783a85d86 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:10.437017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.438114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.440857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2404/4871] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:10 2022] DELETE /identity/v3/roles/db9d4b23c1f244bd93b2266783a85d86 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:10.445510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:10.447049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:10.583425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:10.584489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:10.584957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:10.585426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:10.587497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4960a91-d004-4874-8d9a-fde62218f337 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:10.589571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:10.599686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:10.600842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:10.601771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:10.603722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/cfcab4ef51484975bb132b9c9762c024` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:10.607040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=cfcab4ef51484975bb132b9c9762c024)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:10.611723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:10.680912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.689834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.765453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.766191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:10.773234 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2405/4872] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:10 2022] PATCH /identity/v3/domains/cfcab4ef51484975bb132b9c9762c024 => generated 278 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:10.775576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:10.778594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:10.779643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:10.781680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:10.826581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2468/4873] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:10.941092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0fcd3-8f8c-4408-9c84-b40ee2f1dcc4 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:10.954045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:10.954045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:10.954374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:10.956500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/cfcab4ef51484975bb132b9c9762c024` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:10.979592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=cfcab4ef51484975bb132b9c9762c024)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:10.984285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:11.075444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.082664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:11.108783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.229562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.244247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.263272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.284823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project cfcab4ef51484975bb132b9c9762c024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:11.285842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.288594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.308410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.310017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.318966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'cfcab4ef51484975bb132b9c9762c024', 'request_id': 'req-02e9686d-c269-496a-95f8-abedf571d3c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:11.335352 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2406/4874] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:10:10 2022] DELETE /identity/v3/domains/cfcab4ef51484975bb132b9c9762c024 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:11.341449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:11.358881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4c7af-a359-4f04-9a3e-a9c1f84d570c service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:11.461461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:11.500044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:11.500613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:11.501044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:11.501421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:11.503222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:11.507280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:11.551034 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2469/4875] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:11.555500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:11.557257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:11.588468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9686d-c269-496a-95f8-abedf571d3c1 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:11.606190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:11.607725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:11.608224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:11.608687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/c53745083f98428eba664cb022aff9ef/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:11.613844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c53745083f98428eba664cb022aff9ef, user_id=e0027ae93d7843d1954ee7ba2b201aed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:11.621957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:11.679841 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2407/4876] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:11 2022] PUT /identity/v3/projects/c53745083f98428eba664cb022aff9ef/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:11.684131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:11.703717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534a0e4-6d72-441e-af03-00718947e22b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:11.897013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:11.897805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:11.898173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:11.898361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:11.898983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:11.902096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:12.023705 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2470/4877] 10.222.0.52 () {58 vars in 1146 bytes} [Sat Sep 24 21:10:11 2022] GET /identity/v3/auth/projects => generated 824 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:12.028425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:12.048385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315cb68-6f5c-40be-bb28-dbedb151d884 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:12.066227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:12.066940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:12.067837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:12.068140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/c53745083f98428eba664cb022aff9ef/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:12.074215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=c53745083f98428eba664cb022aff9ef, user_id=e0027ae93d7843d1954ee7ba2b201aed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:12.080084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:12.135678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 49b89b74bd2a42b6ba0c6496570ab8dc was removed from user e0027ae93d7843d1954ee7ba2b201aed on project c53745083f98428eba664cb022aff9ef. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:12.135980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:12.140700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:12.189935 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2408/4878] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:10:12 2022] DELETE /identity/v3/projects/c53745083f98428eba664cb022aff9ef/users/e0027ae93d7843d1954ee7ba2b201aed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:12.195454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:12.203725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:12.232264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145e997-4d7c-462f-8d1a-2561d68de6fe tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:12.401498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:12.402184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:12.402571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:12.402967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:12.403807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:12.421693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:12.513730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2471/4879] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:12 2022] POST /identity/v3/users => generated 321 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:12.525496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:12.539826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09070aa5-0937-4832-8259-bb46ca86cb6b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:12.568067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:12.568810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:12.569161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:12.569505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:12.570217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:12.578082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:12.671253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2409/4880] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:12 2022] POST /identity/v3/projects => generated 364 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:12.680404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:12.696101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee220a2-7f33-4da0-86ba-fba01d746e47 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:12.740365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:12.798595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:12.799366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:12.800210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:12.800210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:12.801217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:12.803401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:12.861332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2472/4881] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:12 2022] POST /identity/v3/projects => generated 364 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:12.865546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:12.882013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f3db6-161a-4d5e-b914-28ea1bca300b tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:13.089010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:13.090992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.091362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.092236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.097324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:13.106546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:13.198617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2410/4882] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:12 2022] POST /identity/v3/roles => generated 239 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:13.202228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:13.232453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc42981-c961-4390-8c26-23184a7b2bfb tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:13.244539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:13.253647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.253982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.254268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/403cf8009de9401eb36204f40694fac2/users/bc9eeadbdb94408d958d40ca1c26dc19/roles/6a9c531fad4b484188bb11c621721d2a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.311495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(project_id=403cf8009de9401eb36204f40694fac2, user_id=bc9eeadbdb94408d958d40ca1c26dc19, role_id=6a9c531fad4b484188bb11c621721d2a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:13.326409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:13.361491 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2473/4883] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:13 2022] PUT /identity/v3/projects/403cf8009de9401eb36204f40694fac2/users/bc9eeadbdb94408d958d40ca1c26dc19/roles/6a9c531fad4b484188bb11c621721d2a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:13.365676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:13.387526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bc85b-436f-49a1-b3ce-8bf684b10fc2 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:13.411201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:13.444986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:13.447372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:13.575257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:13.576554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.577186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.579100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/58239da2f2d94a1b91fa28d805c3153d/users/bc9eeadbdb94408d958d40ca1c26dc19/roles/6a9c531fad4b484188bb11c621721d2a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.583863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:create_grant(project_id=58239da2f2d94a1b91fa28d805c3153d, user_id=bc9eeadbdb94408d958d40ca1c26dc19, role_id=6a9c531fad4b484188bb11c621721d2a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:13.587875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510aab4-4cb6-44fc-9a5e-fe2d363c44e5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:13.627475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2411/4884] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:13 2022] PUT /identity/v3/projects/58239da2f2d94a1b91fa28d805c3153d/users/bc9eeadbdb94408d958d40ca1c26dc19/roles/6a9c531fad4b484188bb11c621721d2a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:13.642132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed7e3f-4a4c-43de-a4db-e6261f004e9a tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:13.650961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0dd43-f447-470e-b47f-b32f415d1c04 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.655850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0dd43-f447-470e-b47f-b32f415d1c04 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.656573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0dd43-f447-470e-b47f-b32f415d1c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.751991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c1e0dd43-f447-470e-b47f-b32f415d1c04 None None] MFA Rules not processed for user `bc9eeadbdb94408d958d40ca1c26dc19`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:13.755002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c1e0dd43-f447-470e-b47f-b32f415d1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:13.762210 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2412/4885] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:13 2022] POST /identity/v3/auth/tokens => generated 334 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:10:13.767152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.767655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.768088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.794129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] MFA Rules not processed for user `bc9eeadbdb94408d958d40ca1c26dc19`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:13.800273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:13.830279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:13.833031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:13.833528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:13.833952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] PATH_INFO: `/v3/users/dc0a3805b2924155a370e40d79091eec` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:13.835872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0a3805b2924155a370e40d79091eec)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:13.845084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:13.976156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:14.007948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0a3805b2924155a370e40d79091eec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:14.009330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0a3805b2924155a370e40d79091eec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:14.021314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2413/4886] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 24 21:10:14.026341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:14.035233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0a3805b2924155a370e40d79091eec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:14.070626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0a3805b2924155a370e40d79091eec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:14.072809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-683c1d88-034b-4f75-bb77-462b1b188823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0a3805b2924155a370e40d79091eec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:14.133625 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2474/4887] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:13 2022] DELETE /identity/v3/users/dc0a3805b2924155a370e40d79091eec => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:14.135616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:14.136310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:14.136817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:14.203562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:14.205414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.223751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e297201-2512-44bc-89aa-b1c8c213318c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:14.373068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:14.439298 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2475/4888] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:14.443394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:14.444048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:14.444307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:14.458206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:14.470272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.497007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:14.499177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.643361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99239827-02d3-4219-8f35-93ca13b0fc7a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:14.653173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:14.653810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:14.654089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:14.654370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:14.670247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:14.672452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:14.675464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:14.676970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2333-adeb-4faa-9eaa-9055aa23f808 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:14.704853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2476/4889] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:14.724073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2414/4890] 10.222.0.52 () {56 vars in 1308 bytes} [Sat Sep 24 21:10:14 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:14.730038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:14.730321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:14.730690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:14.739497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.762981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] MFA Rules not processed for user `bc9eeadbdb94408d958d40ca1c26dc19`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:14.764153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.945456 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2477/4891] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 24 21:10:14.947296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:14.948761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:14.971246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e684f-33e1-455e-8f22-d18c85a09394 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:14.989460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:14.990391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:14.990878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:14.991323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/roles/6a9c531fad4b484188bb11c621721d2a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:14.996168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6a9c531fad4b484188bb11c621721d2a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:15.006098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:15.152848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invalidating the token cache because role 6a9c531fad4b484188bb11c621721d2a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:15.153151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.156619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.169159 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2415/4892] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:14 2022] DELETE /identity/v3/roles/6a9c531fad4b484188bb11c621721d2a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:15.170349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:15.181594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:15.195805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05100909-fe0a-4ffb-8c9d-a0a99eee5d43 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:15.324234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:15.324949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:15.325411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:15.325770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/58239da2f2d94a1b91fa28d805c3153d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:15.328619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_project(project_id=58239da2f2d94a1b91fa28d805c3153d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:15.331476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:15.487333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58239da2f2d94a1b91fa28d805c3153d', 'request_id': 'req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.501997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58239da2f2d94a1b91fa28d805c3153d', 'request_id': 'req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.517244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58239da2f2d94a1b91fa28d805c3153d', 'request_id': 'req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.567319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project 58239da2f2d94a1b91fa28d805c3153d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:15.568051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.581057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:15.583912 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2478/4893] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:15 2022] DELETE /identity/v3/projects/58239da2f2d94a1b91fa28d805c3153d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:15.588426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:15.590556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:15.613420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cc35-8985-4e7b-9cf7-0c5fb4435b58 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:15.832177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:15.832848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:15.833117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:15.833390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/58239da2f2d94a1b91fa28d805c3153d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:15.862590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_project(project_id=58239da2f2d94a1b91fa28d805c3153d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:15.864936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Could not find project: 58239da2f2d94a1b91fa28d805c3153d.: keystone.exception.ProjectNotFound: Could not find project: 58239da2f2d94a1b91fa28d805c3153d. Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 24 21:10:15.898582 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 58239da2f2d94a1b91fa28d805c3153d. Sep 24 21:10:15.900049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:15.904705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2416/4894] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:15 2022] DELETE /identity/v3/projects/58239da2f2d94a1b91fa28d805c3153d => generated 113 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:15.909311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:15.911241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:15.945367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901f465-1ea2-4f1e-81f4-bfc5fc0412a0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:15.983861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:15.984565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:15.990667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:15.994819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/projects/403cf8009de9401eb36204f40694fac2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:15.999639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_project(project_id=403cf8009de9401eb36204f40694fac2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:16.015481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:16.154799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403cf8009de9401eb36204f40694fac2', 'request_id': 'req-00246760-fe69-4794-b62d-926c9a73d8dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.175125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403cf8009de9401eb36204f40694fac2', 'request_id': 'req-00246760-fe69-4794-b62d-926c9a73d8dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.194875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403cf8009de9401eb36204f40694fac2', 'request_id': 'req-00246760-fe69-4794-b62d-926c9a73d8dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.228734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] The token cache is being invalidate because project 403cf8009de9401eb36204f40694fac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:16.229464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.230443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.233231 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2479/4895] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:15 2022] DELETE /identity/v3/projects/403cf8009de9401eb36204f40694fac2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:16.238385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:16.242848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:16.268326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabaed0-3bb6-4ba2-9cf4-deaa5f17fa86 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:16.354332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:16.421693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:16.423110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:16.434994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:16.436299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/users/bc9eeadbdb94408d958d40ca1c26dc19` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:16.442725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorizing `identity:delete_user(user_id=bc9eeadbdb94408d958d40ca1c26dc19)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:16.453144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:16.565362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9eeadbdb94408d958d40ca1c26dc19'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.566257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9eeadbdb94408d958d40ca1c26dc19'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.594946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9eeadbdb94408d958d40ca1c26dc19'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.609731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9eeadbdb94408d958d40ca1c26dc19'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.614042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9eeadbdb94408d958d40ca1c26dc19'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:16.699096 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2417/4896] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:16 2022] DELETE /identity/v3/users/bc9eeadbdb94408d958d40ca1c26dc19 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:16.701844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:16.721088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246760-fe69-4794-b62d-926c9a73d8dc tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:16.811621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:17.019572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:17.020293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.020576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.022762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/c81778c51b1a4f64b5fb333b4561d84e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:17.041253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Could not find domain: c81778c51b1a4f64b5fb333b4561d84e.: keystone.exception.DomainNotFound: Could not find domain: c81778c51b1a4f64b5fb333b4561d84e. Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c81778c51b1a4f64b5fb333b4561d84e. Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:17.051689 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:17.052776 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: c81778c51b1a4f64b5fb333b4561d84e. Sep 24 21:10:17.053792 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.065935 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2480/4897] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:16 2022] PATCH /identity/v3/domains/c81778c51b1a4f64b5fb333b4561d84e => generated 112 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:17.068509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:17.070840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:17.094331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559072ea-4a80-448a-bf67-b3bb76f4bae5 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:17.112964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:17.114009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.114529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.115051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/d79f0f43165c4528a366c2a5d7efcbea` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:17.134900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Could not find domain: d79f0f43165c4528a366c2a5d7efcbea.: keystone.exception.DomainNotFound: Could not find domain: d79f0f43165c4528a366c2a5d7efcbea. Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d79f0f43165c4528a366c2a5d7efcbea. Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:17.150130 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:17.151183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d79f0f43165c4528a366c2a5d7efcbea. Sep 24 21:10:17.152284 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.157864 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2418/4898] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:17 2022] PATCH /identity/v3/domains/d79f0f43165c4528a366c2a5d7efcbea => generated 112 bytes in 90 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:17.169433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:17.197741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf0ac4-5766-4374-94a8-959941384771 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:17.213331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:17.213991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.214277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.214578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/182a27229a8f49e8bd8d8ab2e082a849` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:17.284109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Could not find domain: 182a27229a8f49e8bd8d8ab2e082a849.: keystone.exception.DomainNotFound: Could not find domain: 182a27229a8f49e8bd8d8ab2e082a849. Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 182a27229a8f49e8bd8d8ab2e082a849. Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:17.309047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:17.310218 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 182a27229a8f49e8bd8d8ab2e082a849. Sep 24 21:10:17.315758 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:17.316921 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2481/4899] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:17 2022] PATCH /identity/v3/domains/182a27229a8f49e8bd8d8ab2e082a849 => generated 112 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:17.316994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:17.379702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bd0f-3964-4fec-8017-4448b30d1cce tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:17.409778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0027ae93d7843d1954ee7ba2b201aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68b0301b068a4b149ef23157daf674cf', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:17.411255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.412958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.413970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] PATH_INFO: `/v3/domains/cfcab4ef51484975bb132b9c9762c024` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:17.442224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-4aa0a861-61a3-4bbd-b72e-bc121830daa0 tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Could not find domain: cfcab4ef51484975bb132b9c9762c024.: keystone.exception.DomainNotFound: Could not find domain: cfcab4ef51484975bb132b9c9762c024. Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cfcab4ef51484975bb132b9c9762c024. Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:17.470037 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:17.471095 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: cfcab4ef51484975bb132b9c9762c024. Sep 24 21:10:17.475622 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:17.476851 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2419/4900] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:10:17 2022] PATCH /identity/v3/domains/cfcab4ef51484975bb132b9c9762c024 => generated 112 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:17.476937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:17.478249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:17.693991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db0bb-9aaa-464d-b996-99e7f609e71d tempest-TokensV3TestJSON-235978233 tempest-TokensV3TestJSON-235978233-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:17.705979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:17.707303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.707303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.707507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] PATH_INFO: `/v3/users/7ca1fa6f8d074fcf895eac45141f6ac9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:17.710347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca1fa6f8d074fcf895eac45141f6ac9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:17.713398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:17.908094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca1fa6f8d074fcf895eac45141f6ac9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:17.908900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca1fa6f8d074fcf895eac45141f6ac9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:17.925842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca1fa6f8d074fcf895eac45141f6ac9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:17.967120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca1fa6f8d074fcf895eac45141f6ac9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:17.968052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca1fa6f8d074fcf895eac45141f6ac9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:17.993424 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2482/4901] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:17 2022] DELETE /identity/v3/users/7ca1fa6f8d074fcf895eac45141f6ac9 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:17.997809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:17.998729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:17.999195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.036646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:18.038374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.068329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:18.069931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.081196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:18.159350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e90f65-bff5-4c12-9b19-b232be68be0c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:18.179996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:18.181025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.181559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.182074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] PATH_INFO: `/v3/projects/09b303d9b041415aa4b96dbb7dfe4dae` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.186799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b303d9b041415aa4b96dbb7dfe4dae)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:18.189012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:18.214186 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2420/4902] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:18.282093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:18.283729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.310649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b303d9b041415aa4b96dbb7dfe4dae', 'request_id': 'req-012a2b4a-cb33-41d1-be58-33681650bc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:18.322156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b303d9b041415aa4b96dbb7dfe4dae', 'request_id': 'req-012a2b4a-cb33-41d1-be58-33681650bc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:18.339942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b303d9b041415aa4b96dbb7dfe4dae', 'request_id': 'req-012a2b4a-cb33-41d1-be58-33681650bc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:18.367786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] The token cache is being invalidate because project 09b303d9b041415aa4b96dbb7dfe4dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:18.368977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:18.370657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-012a2b4a-cb33-41d1-be58-33681650bc5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:18.376732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2483/4903] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:18 2022] DELETE /identity/v3/projects/09b303d9b041415aa4b96dbb7dfe4dae => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:18.387316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe00c-0e71-4365-8148-32646c25c011 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.387740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe00c-0e71-4365-8148-32646c25c011 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.388131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe00c-0e71-4365-8148-32646c25c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.418914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:18.435910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2bbbe00c-0e71-4365-8148-32646c25c011 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:18.442394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2bbbe00c-0e71-4365-8148-32646c25c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.481203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed12c7-880f-4ab3-8db5-a1e2435c1bba None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:18.603526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:18.604133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.606851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2484/4904] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:18.612293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.612580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.612857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.628998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.636425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.648349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.660367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:18.665919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:18.671853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:18.684338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2485/4905] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:18.690616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:18.690831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:18.706835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac1e5-5c27-4ef5-8417-518f55707f8f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:18.710767 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2421/4906] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:10:18.719946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:18.720786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.721071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.721346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.722088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:18.722393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:18.724617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:18.757740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:18.768155 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2486/4907] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:18.776339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92872a-d7f2-4a72-b43b-d023212ec21c service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:18.791436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:18.792067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:18.792346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:18.792614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:18.793394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:18.796296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:18.854757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2422/4908] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:18 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:18.854980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:18.872531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3637c70-cf02-41fe-86de-a323e6bc4d0b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:18.985899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:19.043630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.044265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.044543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.044814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.045446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.053564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:19.111060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2487/4909] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:18 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:19.115634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:19.132980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcc328-3f83-4718-b148-4e77fd8bce54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:19.152186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.153105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.153520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.185136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.186281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.189644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:19.211372 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2423/4910] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:19.212599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:19.231378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fd2a1-e84e-445f-b647-9474ea33c241 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:19.245658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.247389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.247681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.248914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.249582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.253836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73792d-25b9-4569-81bd-89160a331325 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:19.278230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2488/4911] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:10:19.279642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:19.300843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a0bb-a2c7-4a47-8555-f05c233d8047 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:19.308289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.309168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.309757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.310209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] PATH_INFO: `/v3/projects/2d33107135914596b2bb2b9e20a4ca0e/users/bb804ca21be04276a3d3e2c930c21f69/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.332333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d33107135914596b2bb2b9e20a4ca0e, user_id=bb804ca21be04276a3d3e2c930c21f69, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.335449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:19.387514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2424/4912] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:19 2022] PUT /identity/v3/projects/2d33107135914596b2bb2b9e20a4ca0e/users/bb804ca21be04276a3d3e2c930c21f69/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:19.387737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.387908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.388186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.446064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] MFA Rules not processed for user `bb804ca21be04276a3d3e2c930c21f69`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:19.461088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:19.646615 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2489/4913] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:19.652779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:19.833229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17976dd6-5446-4437-92eb-5b0ad6fc29e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:19.847711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.848398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.848683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.848956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.849593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.851653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:19.860064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:19.861636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:19.899426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e4345-f83f-4136-8e19-b43b75328341 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:19.945655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:19.951050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:19.951350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:19.951603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] PATH_INFO: `/v3/projects/c53745083f98428eba664cb022aff9ef` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:19.959072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53745083f98428eba664cb022aff9ef)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:19.977356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.029772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2425/4914] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:19 2022] POST /identity/v3/projects => generated 369 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:20.030817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:20.148250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:20.195391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53745083f98428eba664cb022aff9ef', 'request_id': 'req-d6d4af11-728c-4b56-9aa1-1a75e51e2871'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.218244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53745083f98428eba664cb022aff9ef', 'request_id': 'req-d6d4af11-728c-4b56-9aa1-1a75e51e2871'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.237727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53745083f98428eba664cb022aff9ef', 'request_id': 'req-d6d4af11-728c-4b56-9aa1-1a75e51e2871'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.268016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ccfd-f7bf-4a86-bfd6-5a4f652455a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:20.276845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] The token cache is being invalidate because project c53745083f98428eba664cb022aff9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:20.277196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.289152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.295801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2490/4915] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:19 2022] DELETE /identity/v3/projects/c53745083f98428eba664cb022aff9ef => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:20.296741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:20.299161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:20.363582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:20.441526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4af11-728c-4b56-9aa1-1a75e51e2871 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:20.451390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:20.451968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:20.452839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:20.453329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:20.453745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:20.454615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:20.454884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:20.455549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] PATH_INFO: `/v3/users/e0027ae93d7843d1954ee7ba2b201aed` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:20.455924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:20.457342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0027ae93d7843d1954ee7ba2b201aed)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:20.458471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.459975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.502641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2426/4916] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:20 2022] POST /identity/v3/users => generated 291 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:20.507724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:20.509819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:20.543888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825f19-6ee0-462f-a9c6-9c0a12d84fa0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:20.560850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:20.562178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:20.562710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:20.563320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:20.564289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:20.567349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0027ae93d7843d1954ee7ba2b201aed'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.567918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.568819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0027ae93d7843d1954ee7ba2b201aed'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.596052 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2427/4917] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:20.600659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:20.603311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:20.608367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0027ae93d7843d1954ee7ba2b201aed'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.629626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0027ae93d7843d1954ee7ba2b201aed'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.632037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0027ae93d7843d1954ee7ba2b201aed'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:20.657426 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2491/4918] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:20 2022] DELETE /identity/v3/users/e0027ae93d7843d1954ee7ba2b201aed => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:20.658605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754d741-66d8-40e8-9837-b8200dd1e6de admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:20.747592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:20.748422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:20.748771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:20.749119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] PATH_INFO: `/v3/projects/298660e8cc7040598840bedab6f8d270/users/cc564fffcc6c4c3cab9b6e70cb0412eb/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:20.774318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298660e8cc7040598840bedab6f8d270, user_id=cc564fffcc6c4c3cab9b6e70cb0412eb, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:20.781373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.812988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2428/4919] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:20 2022] PUT /identity/v3/projects/298660e8cc7040598840bedab6f8d270/users/cc564fffcc6c4c3cab9b6e70cb0412eb/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:20.816874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:20.833051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba26b-16a3-489f-9492-4d8545872b4a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:20.974249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:20.978039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:20.978321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:20.978694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:20.979366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:20.981931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38404d1a-0e54-4f83-8c25-c74051d67e00 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:20.999720 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2492/4920] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:21.004188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.030251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a6ce-d3c5-4d4e-87fc-a20e6bc8d301 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:21.061571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:21.062431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.066986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.067437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] PATH_INFO: `/v3/projects/298660e8cc7040598840bedab6f8d270/users/cc564fffcc6c4c3cab9b6e70cb0412eb/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.070742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=298660e8cc7040598840bedab6f8d270, user_id=cc564fffcc6c4c3cab9b6e70cb0412eb, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:21.076600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb1adc-08f8-4a8a-a777-b5e0ec2aba0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:21.109833 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2429/4921] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:21 2022] PUT /identity/v3/projects/298660e8cc7040598840bedab6f8d270/users/cc564fffcc6c4c3cab9b6e70cb0412eb/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:21.124037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.124367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.124651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.259535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] MFA Rules not processed for user `cc564fffcc6c4c3cab9b6e70cb0412eb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:21.264599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:21.340112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.340593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.340862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.355927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:21.439699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2493/4922] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:21.442061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] MFA Rules not processed for user `d9c24f9c3b0843fe831eeb8e43b0328a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:21.443592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:21.443919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.473792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c40ae1-5cbd-434e-9c7c-08d9686213e5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:21.477563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:21.501734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:21.503334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.517306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.518339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.520576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:21.522310 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:21.522310 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:21.522310 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:21.526893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:21.554175 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2494/4923] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/services => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:21.570141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.644007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9abaf-e727-4638-935e-4768387c8952 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:21.654432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:21.655119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.658290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.658570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.659266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:21.660288 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:21.660288 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:21.660288 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:21.661230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:21.664480 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2430/4924] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3398 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:21.737930 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1460152012. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:21.737930 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:21.737930 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:21.807037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2495/4925] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/endpoints => generated 361 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:21.810625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.826322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ec6a-745d-4dd9-beb2-e312e5417ded None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:21.865545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:21.866201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.866486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.866827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1460152012` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.867424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1460152012)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:21.869509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:21.874534 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2431/4926] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:10:21 2022] GET /identity/v3/regions/tempest-region-1460152012 => generated 180 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:21.878669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.903899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e28e5-66a3-449c-89ff-0107cfc689d4 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:21.909547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:21.910161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:21.910442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:21.910741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:21.911404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:21.913003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:21.945821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2496/4927] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/services => generated 288 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:21.950116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:21.966592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e875486-fadd-4c35-94d4-adb70fa9bb92 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.027501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.028980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.032033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.033035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.036428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.037540 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:22.037540 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:22.037540 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:22.044965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.125001 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-179738131. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:22.125001 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:22.125001 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:22.197825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2432/4928] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:10:21 2022] POST /identity/v3/endpoints => generated 357 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:22.199774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.225022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646281c-b489-4732-ab48-a8c58bdaa7d9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.263855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.264778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.265547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.265547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-179738131` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.265960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-179738131)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.268092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.284061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2497/4929] 10.222.0.52 () {58 vars in 1203 bytes} [Sat Sep 24 21:10:22 2022] GET /identity/v3/regions/tempest-region-179738131 => generated 178 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:22.285944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.300474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fcc7c-ba71-4c3b-9e40-b1726d2ec7b2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.324001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.324722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.324934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.325180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.332378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.334090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.459048 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-83005233. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:22.459048 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:22.459048 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:22.615691 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2433/4930] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:10:22 2022] POST /identity/v3/endpoints => generated 357 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:22.620051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.662554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97301de-87c9-443e-8a39-bcbc8e7d1b97 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.773567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.775170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.780048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.780048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-83005233` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.780616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-83005233)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.782088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.793973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2498/4931] 10.222.0.52 () {58 vars in 1200 bytes} [Sat Sep 24 21:10:22 2022] GET /identity/v3/regions/tempest-region-83005233 => generated 176 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:22.794898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.820021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1035-0353-450f-99d9-1c147df72fb9 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.835528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.836705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.836705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.837091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.838136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.840021 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:22.840021 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:22.840021 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:22.841688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.873368 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2434/4932] 10.222.0.52 () {58 vars in 1134 bytes} [Sat Sep 24 21:10:22 2022] GET /identity/v3/endpoints => generated 4389 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:22.874982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.896581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec667f-91c5-4ace-8e95-3092c0ca945d tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:22.913495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:22.916795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:22.924865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:22.924865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/daae1280a3e7433e83ffa67521437869` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:22.941320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=daae1280a3e7433e83ffa67521437869)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:22.946775 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:22.946775 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:22.946775 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:22.952466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:22.959916 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2499/4933] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:22 2022] GET /identity/v3/endpoints/daae1280a3e7433e83ffa67521437869 => generated 357 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:22.965599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.965801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:22.967388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:22.991367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2751bc6-ec32-4f0d-8c9a-d328d3764902 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.003771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.005042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.005558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.006776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/daae1280a3e7433e83ffa67521437869` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.026872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=daae1280a3e7433e83ffa67521437869)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.030454 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:23.030454 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:23.030454 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:23.036815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:23.073504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'daae1280a3e7433e83ffa67521437869', 'request_id': 'req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.124494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2500/4934] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:10:22 2022] DELETE /identity/v3/endpoints/daae1280a3e7433e83ffa67521437869 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:23.127015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:23.198201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6aa5-d1ae-4d98-ac29-0e2b26b59e8e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.213762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:23.218316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.220883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.221101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.221348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.221990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.225563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da599dd-a7a1-4456-a83f-8619e3771286 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.226929 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:23.226929 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:23.226929 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:23.237681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:23.280036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.281686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.282076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.285277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] PATH_INFO: `/v3/projects/68b0301b068a4b149ef23157daf674cf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.288722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b0301b068a4b149ef23157daf674cf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.291674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:23.324768 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2501/4935] 10.222.0.52 () {58 vars in 1134 bytes} [Sat Sep 24 21:10:23 2022] GET /identity/v3/endpoints => generated 4045 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:23.325673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:23.387885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcb72a-5bbf-47f7-8c5d-c4bc84f486f2 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.403469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.408298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.410057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.410355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/daae1280a3e7433e83ffa67521437869` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.440593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:10:23.454390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b0301b068a4b149ef23157daf674cf', 'request_id': 'req-69ea8c76-c7b1-453a-90ab-68861ac5105d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.466399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b0301b068a4b149ef23157daf674cf', 'request_id': 'req-69ea8c76-c7b1-453a-90ab-68861ac5105d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.480575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b0301b068a4b149ef23157daf674cf', 'request_id': 'req-69ea8c76-c7b1-453a-90ab-68861ac5105d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Could not find endpoint: daae1280a3e7433e83ffa67521437869.: keystone.exception.EndpointNotFound: Could not find endpoint: daae1280a3e7433e83ffa67521437869. Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._iter().one() Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._only_one_row( Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:23.481147 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: daae1280a3e7433e83ffa67521437869. Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:23.482266 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:10:23.483414 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: daae1280a3e7433e83ffa67521437869. Sep 24 21:10:23.484454 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:23.485857 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2502/4936] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:10:23 2022] DELETE /identity/v3/endpoints/daae1280a3e7433e83ffa67521437869 => generated 114 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:23.488680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:23.498787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] The token cache is being invalidate because project 68b0301b068a4b149ef23157daf674cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:23.499437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.500337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:23.502386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2435/4937] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:22 2022] DELETE /identity/v3/projects/68b0301b068a4b149ef23157daf674cf => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:23.506187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:23.508056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:23.516114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b045a8f-1aad-43e4-b16d-799102151ee7 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.640391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:23.759744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.760580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.764435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.764435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-83005233` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.766981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-83005233)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.791981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea8c76-c7b1-453a-90ab-68861ac5105d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.793291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:23.828681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.829897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.831906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.833464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] PATH_INFO: `/v3/users/5decbb0b75394a6c8678d7f6bd4092cc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.837157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5decbb0b75394a6c8678d7f6bd4092cc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.842776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:23.848849 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-83005233. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:23.848849 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:23.848849 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:23.879793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2503/4938] 10.222.0.52 () {58 vars in 1203 bytes} [Sat Sep 24 21:10:23 2022] DELETE /identity/v3/regions/tempest-region-83005233 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:23.897885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:23.902658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:23.964627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ba4be-8a95-44d9-959a-c678306defaa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:23.973178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:23.974280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:23.974842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:23.975369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:23.976429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:23.982879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.000563 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2504/4939] 10.222.0.52 () {58 vars in 1134 bytes} [Sat Sep 24 21:10:23 2022] GET /identity/v3/endpoints => generated 4045 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:24.008348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.021050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74976c09-7b1c-48aa-b025-e18ae8cd3eb6 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.029260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.030589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.032027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.032563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.033539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Adding query filter params (service_id=5aa4759351204030b1f609d00067d173) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:24.034299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.036271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.049792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2505/4940] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:10:24 2022] GET /identity/v3/endpoints?service_id=5aa4759351204030b1f609d00067d173 => generated 503 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:24.054243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.088157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497041d1-eb52-4c32-8a03-5addb0aeea05 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.123048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.124235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.124853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.125627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.126733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Adding query filter params (service_id=3eb6f1d9b04349b4bb82b018493aaefe) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:24.127699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.131041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:24.135097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:24.146845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.173337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:24.202212 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2506/4941] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:10:24 2022] GET /identity/v3/endpoints?service_id=3eb6f1d9b04349b4bb82b018493aaefe => generated 499 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:24.211778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.219070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:24.223578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:24.228444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5decbb0b75394a6c8678d7f6bd4092cc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:24.279037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10060a-bb37-4d2d-ab0e-a146c65063fa tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.292455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.295998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.296593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.296980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.297773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:24.299410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.301264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.304388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:24.339760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2507/4942] 10.222.0.52 () {58 vars in 1167 bytes} [Sat Sep 24 21:10:24 2022] GET /identity/v3/endpoints?interface=public => generated 3406 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:24.343821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.344324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2436/4943] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:23 2022] DELETE /identity/v3/users/5decbb0b75394a6c8678d7f6bd4092cc => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:24.367620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d685a-ad36-4a39-9089-1ad415e1bd04 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.543765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.544565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.544954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.547586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.548632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:24.549070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.550854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.565444 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2508/4944] 10.222.0.52 () {58 vars in 1171 bytes} [Sat Sep 24 21:10:24 2022] GET /identity/v3/endpoints?interface=internal => generated 474 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:10:24.571037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.588563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65031ecb-6658-4877-a1f6-376009338d50 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.604974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.605677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.611006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.611370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.615757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.620479 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:24.620479 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:24.620479 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:24.621205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.677588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2437/4945] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:24 2022] POST /identity/v3/services => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:24.682752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.697575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6d1-30f8-43d8-83d5-c1afe3ad240c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:24.781800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:24.790243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:24.790616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:24.790876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:24.794478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:24.796242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:24.903281 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1388311490. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:24.903281 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:24.903281 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:24.975117 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2509/4946] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:10:24 2022] POST /identity/v3/endpoints => generated 361 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:24.976455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:24.993577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6d2c-ea3b-4c38-9cc2-cb42cca5b07e tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.048429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.049268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.049724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.050183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1388311490` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.051032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1388311490)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.052261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.059836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.066262 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2438/4947] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:10:24 2022] GET /identity/v3/regions/tempest-region-1388311490 => generated 180 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:25.079348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fef9-9dea-44b2-a299-6a8d53a04609 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.089955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.090858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.091138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.091410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/1b6c362d283a42d8a45a322b3ebd1a27` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.099193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=1b6c362d283a42d8a45a322b3ebd1a27)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.100225 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:25.100225 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:25.100225 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:25.101117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.147790 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1950228871. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:25.147790 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:25.147790 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:25.165631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:25.235158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.235646 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2510/4948] 10.222.0.52 () {60 vars in 1256 bytes} [Sat Sep 24 21:10:25 2022] PATCH /identity/v3/endpoints/1b6c362d283a42d8a45a322b3ebd1a27 => generated 364 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:10:25.255218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edbc82b-cb6f-4255-a4c6-240c8e9b9d4c tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.302928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.303836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.304294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.304744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1950228871` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.306414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1950228871)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.308372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.320480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.326713 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2439/4949] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:10:25 2022] GET /identity/v3/regions/tempest-region-1950228871 => generated 180 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:10:25.338261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43410b1-5864-4c65-89f3-a4a3997bb102 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.358772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.359909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.360360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.360782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/1b6c362d283a42d8a45a322b3ebd1a27` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.377990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=1b6c362d283a42d8a45a322b3ebd1a27)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.379945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.406122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '1b6c362d283a42d8a45a322b3ebd1a27', 'request_id': 'req-1b56ebd9-b24a-49b9-902f-f32043a0a742'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:25.431030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.431676 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2511/4950] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/endpoints/1b6c362d283a42d8a45a322b3ebd1a27 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:25.444049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd68bfe-7f80-4984-9de0-1b9aa67ebd75 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.492266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.493006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.493282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.493949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1950228871` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.494240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1950228871)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.499249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.530773 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1950228871. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:10:25.530773 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:10:25.530773 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:25.546009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.554685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2440/4951] 10.222.0.52 () {58 vars in 1209 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/regions/tempest-region-1950228871 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:25.560483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ebd9-b24a-49b9-902f-f32043a0a742 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.598938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.599845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.600137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.600453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1388311490` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.601088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1388311490)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.603893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.644772 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2512/4952] 10.222.0.52 () {58 vars in 1209 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/regions/tempest-region-1388311490 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:25.650469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.674006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81724782-4a4b-41d8-ab5a-3efe8c4aa652 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.724097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.724695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.725073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.725347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services/43ba4e2e59d349419f0671a6af82d68c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.725935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_service(service_id=43ba4e2e59d349419f0671a6af82d68c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.727039 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:25.727039 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:25.727039 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:25.728288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.784667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.785361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2441/4953] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/services/43ba4e2e59d349419f0671a6af82d68c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:25.797514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296824-1d2f-4585-b8b3-d8879090c3c5 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.818968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.819828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.820394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.821543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/c380d6b034174a6583251aa7f1c6e575` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.829385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=c380d6b034174a6583251aa7f1c6e575)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.832323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:25.854290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'c380d6b034174a6583251aa7f1c6e575', 'request_id': 'req-cb6915f9-292d-481f-b4e5-b61026499d38'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:25.880147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:25.888465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2513/4954] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/endpoints/c380d6b034174a6583251aa7f1c6e575 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:25.905129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9513-e976-422c-9f7f-6fa25c5c2363 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:25.940611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:25.941550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:25.942012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:25.942439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-179738131` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:25.943511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-179738131)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:25.945486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.006540 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2442/4955] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:10:25 2022] DELETE /identity/v3/regions/tempest-region-179738131 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:26.011935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.027524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6915f9-292d-481f-b4e5-b61026499d38 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.069027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.069986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.070432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.071156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services/3eb6f1d9b04349b4bb82b018493aaefe` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.072713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3eb6f1d9b04349b4bb82b018493aaefe)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.073923 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:26.073923 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:26.073923 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:26.075268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.130182 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2514/4956] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/services/3eb6f1d9b04349b4bb82b018493aaefe => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:26.134982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.153724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6397d8-fa3b-40ff-b76e-781210f32cf1 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.218007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.222596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.225106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.226306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/endpoints/b0c47db08f4344f9a26ddb2e7c4d84a1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.271526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=b0c47db08f4344f9a26ddb2e7c4d84a1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.277423 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:26.277423 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:26.277423 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:26.286940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.312188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'b0c47db08f4344f9a26ddb2e7c4d84a1', 'request_id': 'req-728f5ee4-d639-467c-a09d-f6572dc538eb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.329360 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2443/4957] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/endpoints/b0c47db08f4344f9a26ddb2e7c4d84a1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:26.333550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.351356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6e4b3-392e-45fc-9562-8984099e5d07 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.381863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.385025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:26.400032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.401069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.402006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.402442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/regions/tempest-region-1460152012` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.404928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1460152012)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.409155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.479828 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2515/4958] 10.222.0.52 () {58 vars in 1209 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/regions/tempest-region-1460152012 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:26.481220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.522888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:26.536881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9112-bb42-437f-acc0-6f39e9ba7c81 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.560058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc564fffcc6c4c3cab9b6e70cb0412eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '298660e8cc7040598840bedab6f8d270', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.561744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.562039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.562383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] PATH_INFO: `/v3/services/5aa4759351204030b1f609d00067d173` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.563016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorizing `identity:delete_service(service_id=5aa4759351204030b1f609d00067d173)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.564817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.587862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f5ee4-d639-467c-a09d-f6572dc538eb tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.622547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.624642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.625020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.625194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] PATH_INFO: `/v3/projects/291b5747bb35450aaeedfde7912efefd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.630552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=291b5747bb35450aaeedfde7912efefd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.632570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:26.654460 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2516/4959] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/services/5aa4759351204030b1f609d00067d173 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:26.659023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:26.662286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:26.743940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291b5747bb35450aaeedfde7912efefd', 'request_id': 'req-b124828d-1895-4a5a-a214-0960ccfda43f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.766816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291b5747bb35450aaeedfde7912efefd', 'request_id': 'req-b124828d-1895-4a5a-a214-0960ccfda43f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.780623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291b5747bb35450aaeedfde7912efefd', 'request_id': 'req-b124828d-1895-4a5a-a214-0960ccfda43f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.801798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] The token cache is being invalidate because project 291b5747bb35450aaeedfde7912efefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:26.802229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.804878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b124828d-1895-4a5a-a214-0960ccfda43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:26.806332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97820-60be-4e67-b972-10beb204fdb8 tempest-EndPointsTestJSON-828925770 tempest-EndPointsTestJSON-828925770-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:26.813937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2444/4960] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/projects/291b5747bb35450aaeedfde7912efefd => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:26.815567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e2c4-7bee-4801-9767-debbde53f0cb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.815847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e2c4-7bee-4801-9767-debbde53f0cb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.816124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e2c4-7bee-4801-9767-debbde53f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.829376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:UPDATE=1,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:26.888351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74f5e2c4-7bee-4801-9767-debbde53f0cb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:26.889593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74f5e2c4-7bee-4801-9767-debbde53f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:26.972982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:26.974588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:26.975160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:26.976041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] PATH_INFO: `/v3/users/bb804ca21be04276a3d3e2c930c21f69` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:26.979711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb804ca21be04276a3d3e2c930c21f69)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:26.984650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.080519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2445/4961] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:27.082223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.082675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.082983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.115565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb804ca21be04276a3d3e2c930c21f69'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:27.119973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb804ca21be04276a3d3e2c930c21f69'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:27.132558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:27.135281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:27.139486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb804ca21be04276a3d3e2c930c21f69'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:27.161930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb804ca21be04276a3d3e2c930c21f69'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:27.165936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eed3c336-4355-46df-938b-1367eda401d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb804ca21be04276a3d3e2c930c21f69'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:27.168083 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2446/4962] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:27.175155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.195086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94277f7-c2a0-4cb9-a688-9cffee832ba3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.202451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2517/4963] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:26 2022] DELETE /identity/v3/users/bb804ca21be04276a3d3e2c930c21f69 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:27.213300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.213560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.213844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.262230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:27.263699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:27.337010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:27.354362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.356152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.356432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.356708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.361062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:27.362464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.364388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.402967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2447/4964] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:27.403909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.429282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef22fa-759d-4510-89ba-c1047ffea930 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.438390 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2518/4965] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:27.441429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.443111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.443458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.443781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.444494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.447544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.481657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.483682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:27.512265 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2448/4966] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:27 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:27.520585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.595402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:27.696209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d34d-a29e-44cd-8c8f-3628bb13dd58 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.713063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ccdd-c113-4d35-9dde-f72aac450ac2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.726081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.728867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.729162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.729440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.734171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.736507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.809572 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2449/4967] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:27 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:27.810619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.835996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b9433-0613-4537-bc5f-7f482a4c659a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.840606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.841390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.841668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.842250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.847577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.848376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.848661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.848935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.850421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.855244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.869210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.872524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.892648 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2450/4968] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:27.897617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:27.919451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2519/4969] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:10:27.931929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db5651-a384-4c13-b745-46d2681ed3b4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:27.947429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:27.948103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:27.948391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:27.948665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:27.949315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:27.952912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3eb1aa-8907-4a7a-b760-16375b165629 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:27.980333 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2451/4970] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:27.985044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:28.011701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213b14a-2ddf-4297-a7b5-d1a268a4776d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:28.028861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:28.029541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:28.029880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:28.030346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] PATH_INFO: `/v3/projects/563286e399024868b69af46757a5ffaf/users/a955cdcb010e44c0bcd3b03256b7f76a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:28.100002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=563286e399024868b69af46757a5ffaf, user_id=a955cdcb010e44c0bcd3b03256b7f76a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:28.105408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:28.146276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2520/4971] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:27 2022] PUT /identity/v3/projects/563286e399024868b69af46757a5ffaf/users/a955cdcb010e44c0bcd3b03256b7f76a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:28.157981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:28.158318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:28.158563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:28.259003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] MFA Rules not processed for user `a955cdcb010e44c0bcd3b03256b7f76a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:28.260609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:28.516593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:28.517650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2452/4972] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:28.535697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d516648-0bb3-40dc-84ba-ccf8c0f39fba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:28.557903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:28.558702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:28.558985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:28.559391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:28.560376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:28.565047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948608c5-60ed-4d6c-b908-6328d75803a8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:28.847862 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2521/4973] 10.222.0.52 () {60 vars in 1314 bytes} [Sat Sep 24 21:10:28 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 789 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:28.852954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:28.865425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f88e-ef75-4902-b742-6876cd7fa121 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:28.881080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:28.882183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:28.882727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:28.883256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/cd6b956f7a3b4712800e8657f4e9aa74` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:28.908854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:get_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=cd6b956f7a3b4712800e8657f4e9aa74)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:28.911188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:28.919693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:28.920385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:28.920869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:28.925657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2453/4974] 10.222.0.52 () {58 vars in 1392 bytes} [Sat Sep 24 21:10:28 2022] GET /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/cd6b956f7a3b4712800e8657f4e9aa74 => generated 689 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:28.958858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] MFA Rules not processed for user `a955cdcb010e44c0bcd3b03256b7f76a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:28.961782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:29.092890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:29.246434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:29.246882 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2522/4975] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:28 2022] POST /identity/v3/auth/tokens => generated 3427 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) Sep 24 21:10:29.256702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a6e77-dad6-4ebc-a768-58c5a5c8271a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:29.271713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:29.272428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:29.272711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:29.272984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/cd6b956f7a3b4712800e8657f4e9aa74` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:29.276127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=cd6b956f7a3b4712800e8657f4e9aa74)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:29.277696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:29.286617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:29.322682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2454/4976] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:29 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/cd6b956f7a3b4712800e8657f4e9aa74 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:29.328682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:29.339157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b51ad4-e21b-457f-a11d-5287186b8054 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:29.353719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:29.354416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:29.355093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:29.355093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:29.355671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:29.356855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:29.616707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:29.622753 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2523/4977] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:10:29 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 921 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:29.628399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960217c-bcb5-4e97-bdc6-7c0a32f29dc0 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:29.650884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:29.651641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:29.651923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:29.652199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/5f29dc71bff94e55902e1ea3efa09d9f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:29.671973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=5f29dc71bff94e55902e1ea3efa09d9f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:29.674322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:29.725893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2455/4978] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:29 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/5f29dc71bff94e55902e1ea3efa09d9f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:29.731774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:29.747620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd468e8e-3d30-4331-88f5-50753642075b tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:29.760703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:29.761782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:29.762307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:29.762818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:29.763653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:29.767238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.022053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2524/4979] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:10:29 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 813 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:30.027955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.047932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ce167-374e-493d-a14e-07592d880c4a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.076592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.078094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.080746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.081019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/0dd392b3d1374decb5be9e277a9b769b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.128858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=0dd392b3d1374decb5be9e277a9b769b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.136537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.215684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2456/4980] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:30 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/0dd392b3d1374decb5be9e277a9b769b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:30.216614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.226546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb6133-22d2-4ab0-bfb7-18dd97b3f826 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.245445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.248747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.251471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.252783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.254973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.256211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.380113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:30.495889 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2525/4981] 10.222.0.52 () {60 vars in 1314 bytes} [Sat Sep 24 21:10:30 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 788 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:30.497724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.511035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3231-d060-4e1a-9398-aec91962c932 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.527863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.529165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.529697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.530181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.531381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.532788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.741570 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2457/4982] 10.222.0.52 () {60 vars in 1314 bytes} [Sat Sep 24 21:10:30 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 789 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:30.743361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.764350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98804f36-cae4-4548-a3c6-285d1d11aa9a tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.769143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.769815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.770099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.770420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.771041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:list_application_credentials(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.772956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.806038 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2526/4983] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:10:30 2022] GET /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 1501 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:30.810772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.822235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f71e7-8968-4f4a-a7f2-289195e524e3 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.836883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.837563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.837846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.838119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/5c585e3314a94918916025b1c6c98f34` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.861550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=5c585e3314a94918916025b1c6c98f34)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.862244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.864189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:30.866759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:30.915255 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2458/4984] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:30 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/5c585e3314a94918916025b1c6c98f34 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:30.924823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:30.944447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48017cc8-40aa-4300-8f77-5b43892ec8c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:30.952723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:30.953828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:30.954321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:30.955998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/3dd23908ded6452c8b4477b1cfb47c4d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:30.972221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=3dd23908ded6452c8b4477b1cfb47c4d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:30.976938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.060930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2459/4985] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:30 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/3dd23908ded6452c8b4477b1cfb47c4d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:31.061669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fb62e-282e-4db5-a20e-942f33fe905f tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.067398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:31.083779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.085488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.085764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.087415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] PATH_INFO: `/v3/projects/2d33107135914596b2bb2b9e20a4ca0e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.091659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d33107135914596b2bb2b9e20a4ca0e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.096564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.098069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7053c3-cd65-4a71-8bc6-6fa73e8f62f8 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.109271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.109930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.112577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.112892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.114727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.116212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.179648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:31.322828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d33107135914596b2bb2b9e20a4ca0e', 'request_id': 'req-66d0f22d-0d96-4bd9-902f-247661e0113c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:31.374387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d33107135914596b2bb2b9e20a4ca0e', 'request_id': 'req-66d0f22d-0d96-4bd9-902f-247661e0113c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:31.418762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d33107135914596b2bb2b9e20a4ca0e', 'request_id': 'req-66d0f22d-0d96-4bd9-902f-247661e0113c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:31.424877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2460/4986] 10.222.0.52 () {60 vars in 1314 bytes} [Sat Sep 24 21:10:31 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 788 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:31.426039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:31.468691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134cd9-73d3-4d71-99a2-58bd49e62b94 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.489008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] The token cache is being invalidate because project 2d33107135914596b2bb2b9e20a4ca0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:31.489423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:31.491752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:31.500318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2527/4987] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:30 2022] DELETE /identity/v3/projects/2d33107135914596b2bb2b9e20a4ca0e => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:31.503767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:31.510102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:31.664041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.665925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.666429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.666969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.667870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:create_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.669305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.733184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:31.810736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2461/4988] 10.222.0.52 () {60 vars in 1314 bytes} [Sat Sep 24 21:10:31 2022] POST /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials => generated 788 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:31.812043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:31.813979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:31.844439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aeb8ea-139a-46db-a869-650cd75e2748 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.850159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.851259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.852335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.852841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.853914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Adding query filter params (name=tempest-application_credential-872462031) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:31.854456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:list_application_credentials(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.858295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.873757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2462/4989] 10.222.0.52 () {58 vars in 1384 bytes} [Sat Sep 24 21:10:31 2022] GET /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials?name=tempest-application_credential-872462031 => generated 885 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:31.878063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:31.892185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0f22d-0d96-4bd9-902f-247661e0113c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.893238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340777-5ba6-4876-98f2-e1a872bbe4c2 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:31.918089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.920475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.921251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:31.921694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.922076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/1b6b8cebdddf4f68a735a02ed374563f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.922839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:31.923773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:31.924356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] PATH_INFO: `/v3/users/cc564fffcc6c4c3cab9b6e70cb0412eb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:31.928339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc564fffcc6c4c3cab9b6e70cb0412eb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.937936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:31.947002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=1b6b8cebdddf4f68a735a02ed374563f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:31.948996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:32.010249 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2463/4990] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:31 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/1b6b8cebdddf4f68a735a02ed374563f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:32.014164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:32.043148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162916b6-2618-441a-bfbd-92a9bba144f5 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:32.055510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a955cdcb010e44c0bcd3b03256b7f76a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '563286e399024868b69af46757a5ffaf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:32.056211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:32.056476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:32.056750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/449bbd1f29a9428c840c5a20151da6ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:32.081617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorizing `identity:delete_application_credential(user_id=a955cdcb010e44c0bcd3b03256b7f76a, application_credential_id=449bbd1f29a9428c840c5a20151da6ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:32.083322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:32.084171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc564fffcc6c4c3cab9b6e70cb0412eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.087785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc564fffcc6c4c3cab9b6e70cb0412eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.097740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:32.129360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc564fffcc6c4c3cab9b6e70cb0412eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.137751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2464/4991] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:10:32 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a/application_credentials/449bbd1f29a9428c840c5a20151da6ad => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:32.144508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:32.146398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:32.174011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47ce5f-5681-4fd7-a286-819c44f90456 tempest-ApplicationCredentialsV3Test-1758755561 tempest-ApplicationCredentialsV3Test-1758755561-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:32.175107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc564fffcc6c4c3cab9b6e70cb0412eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.176501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-796ff7d6-3368-435d-9765-8e9cb0e950df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc564fffcc6c4c3cab9b6e70cb0412eb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.185642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:32.186260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:32.186572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:32.186929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] PATH_INFO: `/v3/users/a955cdcb010e44c0bcd3b03256b7f76a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:32.189708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a955cdcb010e44c0bcd3b03256b7f76a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:32.194527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:32.238522 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2528/4992] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:31 2022] DELETE /identity/v3/users/cc564fffcc6c4c3cab9b6e70cb0412eb => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:32.367044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a955cdcb010e44c0bcd3b03256b7f76a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.369686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a955cdcb010e44c0bcd3b03256b7f76a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.389796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a955cdcb010e44c0bcd3b03256b7f76a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.400112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a955cdcb010e44c0bcd3b03256b7f76a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.401135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a955cdcb010e44c0bcd3b03256b7f76a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:32.442747 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2465/4993] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:32 2022] DELETE /identity/v3/users/a955cdcb010e44c0bcd3b03256b7f76a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:32.444385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:32.445117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:32.445539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:32.511696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:32.515615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:32.656130 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2529/4994] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:32.669028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:32.671036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:32.886360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14373319-1a15-4b91-928c-b964bf94005d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:32.897326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:32.897994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:32.898270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:32.899732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:32.926884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:32.931668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a4cca-4cfc-4c22-8f3f-adcb519b3490 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:32.976134 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2466/4995] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:34.265582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:34.268759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:34.291893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba52137-9e68-4946-9ced-a70de3a206fd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:34.318075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:34.318765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:34.319048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:34.319324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] PATH_INFO: `/v3/projects/298660e8cc7040598840bedab6f8d270` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:34.327910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] RBAC: Authorizing `identity:delete_project(project_id=298660e8cc7040598840bedab6f8d270)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:34.329813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:34.498913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298660e8cc7040598840bedab6f8d270', 'request_id': 'req-985d63eb-e651-43b7-906e-6727f1b5328f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:34.501553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:34.516790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298660e8cc7040598840bedab6f8d270', 'request_id': 'req-985d63eb-e651-43b7-906e-6727f1b5328f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:34.547257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298660e8cc7040598840bedab6f8d270', 'request_id': 'req-985d63eb-e651-43b7-906e-6727f1b5328f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:34.578907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] The token cache is being invalidate because project 298660e8cc7040598840bedab6f8d270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:34.579860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:34.585352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-985d63eb-e651-43b7-906e-6727f1b5328f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:34.592378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2530/4996] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:34 2022] DELETE /identity/v3/projects/298660e8cc7040598840bedab6f8d270 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:34.597410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:34.598243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:34.598926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:34.643602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:34.645320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:34.786650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:34.813282 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2467/4997] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:34.823338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:34.824278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:34.824278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:34.832345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:34.833862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:34.854282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b337ce2-e63c-4b73-b086-b43edbe31e3d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:34.870451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:34.871130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:34.871408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:34.871678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] PATH_INFO: `/v3/projects/563286e399024868b69af46757a5ffaf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:34.872874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=563286e399024868b69af46757a5ffaf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:34.874841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:34.908459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:34.911213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:34.935812 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2531/4998] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:34.940048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:34.972127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9680-ff19-4813-9550-57b438b9dbad None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.013027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '563286e399024868b69af46757a5ffaf', 'request_id': 'req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:35.029941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '563286e399024868b69af46757a5ffaf', 'request_id': 'req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:35.055552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '563286e399024868b69af46757a5ffaf', 'request_id': 'req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:35.080305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] The token cache is being invalidate because project 563286e399024868b69af46757a5ffaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:35.080606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:35.081394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e8ec4c0-d4d1-4bcd-863e-84bf753de164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:35.084182 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2468/4999] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:34 2022] DELETE /identity/v3/projects/563286e399024868b69af46757a5ffaf => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:35.087226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.088180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.088467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.088739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.090934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:35.091456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.092566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94847d-bcd4-44db-a5bb-ab5c50d28d4a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.092566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94847d-bcd4-44db-a5bb-ab5c50d28d4a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.092753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94847d-bcd4-44db-a5bb-ab5c50d28d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.095749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.128976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2532/5000] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:35.134884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.136244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2d94847d-bcd4-44db-a5bb-ab5c50d28d4a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:35.137350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:35.137470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d94847d-bcd4-44db-a5bb-ab5c50d28d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:35.154891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d984e5d-047d-4648-b10d-ecf88e2e880d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.163839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.165963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.166467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.166988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.168420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.170798 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2469/5001] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:35.172257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.172545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.172691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.173340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.223622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:35.224872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:35.245225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2470/5002] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:35.259051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.259914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2533/5003] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:35.261058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.269461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f4a06-6ab2-43e6-af5e-a889cb210d01 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.369443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16d4f9-c099-48a8-a051-db4341e7876f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.377557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.378751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.379253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.379711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.380003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.380314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.380724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.381002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.381816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:35.381946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.383689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.384315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.387261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.400611 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2471/5004] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:35.401656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.417997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ade70-3666-48d9-9afa-c3d9815efcef admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.424718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.425896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.425896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.426408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.426988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.429621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.437255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2534/5005] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:35.439447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.463928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e12b-df26-46bf-bd5c-3663a6bcd4c3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.472962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.473603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.473887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.474164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.474934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.477345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.480296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2472/5006] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/projects => generated 395 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:35.488173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.504750 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2535/5007] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:35.508332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.532255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73668-9214-4d93-bccc-7300d9b45af3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.572422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:35.644080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:35.659115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.660881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.660881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.660881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.660881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.663838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.701661 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2536/5008] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:35.703319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.748997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546a2e8-184d-414a-a9cb-0388b35d277a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.754268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.754980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.755261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.755536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] PATH_INFO: `/v3/projects/3dc1d2f5cb904a11a86f9d691ea3df3a/users/7d393a3627d84849b55a6b9363c3e237/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.763825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503e8c2-db16-476f-8375-cf15b94ef7f7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:35.771183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:35.774684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.774915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.775368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.788703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc1d2f5cb904a11a86f9d691ea3df3a, user_id=7d393a3627d84849b55a6b9363c3e237, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.791737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d936596-0d3b-44cc-9518-94bc02700779 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.796210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:35.801579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:35.827846 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2537/5009] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:35 2022] PUT /identity/v3/projects/3dc1d2f5cb904a11a86f9d691ea3df3a/users/7d393a3627d84849b55a6b9363c3e237/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:35.839753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:35.840049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:35.840321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:35.878776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2473/5010] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/users => generated 304 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:35.886241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:35.901799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] MFA Rules not processed for user `7d393a3627d84849b55a6b9363c3e237`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:35.903080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:35.907486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da26fce-e994-405f-ba73-f6b5b86c13c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.031626 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2538/5011] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:36.036724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.046051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.047106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.047605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.048186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.049063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.051043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.066490 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2474/5012] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:36.067753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.080775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30328f7c-0de1-4446-a2a9-f79259a04c03 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.100751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.103169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.103693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.104170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] PATH_INFO: `/v3/projects/2390e824cdda4bf98f61da52dcd574e7/users/a19401f560ba4b089a9a652f90e445ed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.145340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2390e824cdda4bf98f61da52dcd574e7, user_id=a19401f560ba4b089a9a652f90e445ed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.151763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.178532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026350-d9e6-4bd9-9f92-71b10473e9fa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.183449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2475/5013] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:36 2022] PUT /identity/v3/projects/2390e824cdda4bf98f61da52dcd574e7/users/a19401f560ba4b089a9a652f90e445ed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:36.187779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.264818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:36.319403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.321093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.321484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.321850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.322624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.324901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.412190 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2539/5014] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:36 2022] POST /identity/v3/projects => generated 385 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:36.412791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.415326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:36.425577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae84d1-8b55-46ef-90be-bc6cba618f86 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.535119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:36.563873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3970c4-494b-406d-96d2-345a9045e35f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.567978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.569062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.569586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.570046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.570822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.571618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.571902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.572180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] PATH_INFO: `/v3/users/d9c24f9c3b0843fe831eeb8e43b0328a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.573432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c24f9c3b0843fe831eeb8e43b0328a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.574159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.575655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.578656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.606886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2476/5015] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:36.607839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.632184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca3d3c-5e6e-42e1-a21a-3f5147cfbf1f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.659892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.661431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.661905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.662334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.663183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.664678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c24f9c3b0843fe831eeb8e43b0328a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:36.665577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c24f9c3b0843fe831eeb8e43b0328a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:36.666746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.676595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c24f9c3b0843fe831eeb8e43b0328a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:36.708465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c24f9c3b0843fe831eeb8e43b0328a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:36.710471 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2477/5016] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:36 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:36.710969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c24f9c3b0843fe831eeb8e43b0328a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:36.712945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.718706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:36.739964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2540/5017] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:36 2022] DELETE /identity/v3/users/d9c24f9c3b0843fe831eeb8e43b0328a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:36.739964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.739964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:36.747096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f56b94-6ce8-4317-aa9b-1ba6b3e8f639 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.824203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:36.860182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b6e2c-3066-454c-8d3f-174bdafefd30 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.871545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.872599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.873039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.873469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.877018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:36.886905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:36.887793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.888160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.888557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] PATH_INFO: `/v3/projects/2390e824cdda4bf98f61da52dcd574e7/users/a19401f560ba4b089a9a652f90e445ed/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.891179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2390e824cdda4bf98f61da52dcd574e7, user_id=a19401f560ba4b089a9a652f90e445ed, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.894625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbc3cd-2a81-4e19-9cf0-3d53b736a03f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.895250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:36.897804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:36.911693 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2541/5018] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:36.913076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:36.915887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:36.916352 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2478/5019] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:36 2022] PUT /identity/v3/projects/2390e824cdda4bf98f61da52dcd574e7/users/a19401f560ba4b089a9a652f90e445ed/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:36.928929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:36.930551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:36.931280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:36.939204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b54d49-0c31-4df2-83e7-28232b1146c8 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:36.967762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] MFA Rules not processed for user `a19401f560ba4b089a9a652f90e445ed`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:36.970215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:37.008371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.009501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.009501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.009837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.010356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.012487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.027208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2542/5020] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:37.031722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.045237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f40e-27ea-4ce7-9ce9-b65f1f3384b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.051545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.052228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.052506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.052780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] PATH_INFO: `/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999/users/081e15c6bdd54c02888aa662ee032b2d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.081270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2479/5021] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:36 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:10:37.086146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.087430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdab24010d9f4e5b8f62a77a0ccfd999, user_id=081e15c6bdd54c02888aa662ee032b2d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.090164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.121215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97732c5f-6aba-43da-8b96-d7a709108258 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.124023 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2543/5022] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:37 2022] PUT /identity/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999/users/081e15c6bdd54c02888aa662ee032b2d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:37.129979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.198061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:37.221569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.223094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.224274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.225488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.226818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.229883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.272777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281011-b00f-4e36-98f6-2c83433efdd2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.288984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2480/5023] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:37 2022] POST /identity/v3/projects => generated 397 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:37.292213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.292741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.292864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.293138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.293507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.294162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.296258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.310547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2544/5024] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:37.314272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.333908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bc573-d7c5-4474-aa31-ad057d6593fe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.390177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f093f5-79c0-4683-802c-4cb3e598aebd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.395519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.396222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.397134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.397561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.398949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.401667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.424567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:37.442682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2481/5025] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:37 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:37.443911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.453984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.454851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.455261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.455648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] PATH_INFO: `/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999/users/081e15c6bdd54c02888aa662ee032b2d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.458165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdab24010d9f4e5b8f62a77a0ccfd999, user_id=081e15c6bdd54c02888aa662ee032b2d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.461429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dafd89-c2ae-439e-811b-47c1641e232f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.463696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aac5d-276e-4fdf-b3fa-2366197b99f6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.475027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2545/5026] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:37 2022] PUT /identity/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999/users/081e15c6bdd54c02888aa662ee032b2d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:37.482445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.487654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.488068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.488447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.515449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.518568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.518879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.526972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.535786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.569585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] MFA Rules not processed for user `081e15c6bdd54c02888aa662ee032b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:37.573612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:37.579396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2482/5027] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:37.580498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.585086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:37.614059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642c128-ffec-413e-962f-0fe00360550a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.699752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.701348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.701999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.702460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] PATH_INFO: `/v3/projects/7137f08b68d046bc9017aa466b7da5a9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.705773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7137f08b68d046bc9017aa466b7da5a9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.709494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.768449 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2546/5028] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:37.769586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.790894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bff164-41cc-4253-aefe-7b98e6e7a417 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:37.800372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7137f08b68d046bc9017aa466b7da5a9', 'request_id': 'req-bab84334-18a4-4808-bbb5-5da4c48eae42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:37.811363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:37.817528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7137f08b68d046bc9017aa466b7da5a9', 'request_id': 'req-bab84334-18a4-4808-bbb5-5da4c48eae42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:37.839052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7137f08b68d046bc9017aa466b7da5a9', 'request_id': 'req-bab84334-18a4-4808-bbb5-5da4c48eae42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:37.860728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] The token cache is being invalidate because project 7137f08b68d046bc9017aa466b7da5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:37.861003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:37.862541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:37.865324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2483/5029] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:37 2022] DELETE /identity/v3/projects/7137f08b68d046bc9017aa466b7da5a9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:37.873180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:37.875698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:37.932436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:37.933444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.933444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.933669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:37.934222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:37.936208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd9cb5-9f88-4848-a3ea-02b193e205ff admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:37.964323 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2547/5030] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:37.964323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:37.964323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:37.964323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.003517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:38.006943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.034700 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2548/5031] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:38.034700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.037347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.053524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36abde-a0d9-4813-a652-2c38cded2aea None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.073386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.074080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.074358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.075442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] PATH_INFO: `/v3/projects/009521c189ab4b0ca2b073960b7ab359/users/b2ea0b9151df4b6cbb53f3d07b2c1922/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.106614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009521c189ab4b0ca2b073960b7ab359, user_id=b2ea0b9151df4b6cbb53f3d07b2c1922, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.109323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5d056-4435-4f83-a96f-c5fc52068343 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:38.128288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab84334-18a4-4808-bbb5-5da4c48eae42 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.148999 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2549/5032] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:38 2022] PUT /identity/v3/projects/009521c189ab4b0ca2b073960b7ab359/users/b2ea0b9151df4b6cbb53f3d07b2c1922/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:38.148999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.148999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.148999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.192038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:38.193574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.235105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '081e15c6bdd54c02888aa662ee032b2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdab24010d9f4e5b8f62a77a0ccfd999', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.236120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.236598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.237097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.237944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.240709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc70f1d-3968-4415-9a9d-603d4d25dec6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:38.256764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:38.301144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2484/5033] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:37 2022] POST /identity/v3/projects => generated 360 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:38.302721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.303010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.303284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.340085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] MFA Rules not processed for user `b2ea0b9151df4b6cbb53f3d07b2c1922`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:38.341738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.343178 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2550/5034] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:38.346110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.348187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.435222 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2485/5035] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:38.436473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.441197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:38.497606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6399e83-82cd-449d-b8d5-e6467e6d6fad None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.517282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc3bbb-8620-4542-a70b-975f7b90fb9e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.522468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.523721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.524015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.524764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.525477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:38.525790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.527836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:38.547802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2486/5036] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:38.547802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.548741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.576658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d393a3627d84849b55a6b9363c3e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc1d2f5cb904a11a86f9d691ea3df3a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.577878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.578742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.579869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] PATH_INFO: `/v3/projects/061db7b7ccdd4a09ac62898ae791c8a4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.581767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] RBAC: Authorizing `identity:delete_project(project_id=061db7b7ccdd4a09ac62898ae791c8a4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete Sep 24 21:10:38.591317 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. Sep 24 21:10:38.592456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:38.594321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2551/5037] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:38 2022] DELETE /identity/v3/projects/061db7b7ccdd4a09ac62898ae791c8a4 => generated 136 bytes in 249 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:38.595418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.612073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150fc63-e1d4-4ccd-b8da-c93ce91e4cc4 tempest-ProjectsNegativeTestJSON-1847348039 tempest-ProjectsNegativeTestJSON-1847348039-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.626052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.626859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.627210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.627549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.628580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.647548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:38.741061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2552/5038] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:38 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:38.741061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:38.766293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce401546-2430-4e45-bf23-edb6e107fd15 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.846660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7fb41-9756-4db7-b5cf-b43185c7f248 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:38.854763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '081e15c6bdd54c02888aa662ee032b2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdab24010d9f4e5b8f62a77a0ccfd999', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.856089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.856390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.856648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] PATH_INFO: `/v3/projects/061db7b7ccdd4a09ac62898ae791c8a4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.858113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: Authorizing `identity:delete_project(project_id=061db7b7ccdd4a09ac62898ae791c8a4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:38.863503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:38.863503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:38.863503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:38.863503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:38.864728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:38.869684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:38.884652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:38.943278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061db7b7ccdd4a09ac62898ae791c8a4', 'request_id': 'req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:38.956827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061db7b7ccdd4a09ac62898ae791c8a4', 'request_id': 'req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:38.969123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061db7b7ccdd4a09ac62898ae791c8a4', 'request_id': 'req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:38.997809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] The token cache is being invalidate because project 061db7b7ccdd4a09ac62898ae791c8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:38.998150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:38.999354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.006965 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2553/5039] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:38 2022] DELETE /identity/v3/projects/061db7b7ccdd4a09ac62898ae791c8a4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:39.007937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.010391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:39.011227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.027863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.029495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.047987 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2487/5040] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:39.049781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.064150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.123817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421230-5239-4c4c-865a-a6c7cb1bd9c6 tempest-ProjectsNegativeTestJSON-2128755671 tempest-ProjectsNegativeTestJSON-2128755671-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.136754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.137381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.137662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.139858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.140527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.149161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.198047 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2554/5041] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:39 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:39.203597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.212535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79111529-cd59-48a8-9e16-4467e1b05ac5 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.217360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e320c-152d-4fa0-84f7-34b9be6808e5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.223631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.224280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.224562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.224838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] PATH_INFO: `/v3/users/7d393a3627d84849b55a6b9363c3e237` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.226335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d393a3627d84849b55a6b9363c3e237)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.228562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.231777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.232408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.232690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.232965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.233715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.235265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.263013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2555/5042] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:39.267223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.294390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272e68a-7898-47e9-8585-adccfba3212d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.305436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.306138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.306590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.307086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.307738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.309358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.314068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d393a3627d84849b55a6b9363c3e237'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.315049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d393a3627d84849b55a6b9363c3e237'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.326442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d393a3627d84849b55a6b9363c3e237'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.344328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d393a3627d84849b55a6b9363c3e237'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.345641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-01358fa6-8598-4d88-90af-e6b87d2b0a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d393a3627d84849b55a6b9363c3e237'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:39.358557 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2556/5043] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:39.359949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.362228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.379755 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2488/5044] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:39 2022] DELETE /identity/v3/users/7d393a3627d84849b55a6b9363c3e237 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:39.384370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.385295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.385716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.407032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45389a-9be0-4355-a62e-d434897767a4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.412229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:39.414039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.521925 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2489/5045] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:39.525568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.526265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.526639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.526919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] PATH_INFO: `/v3/projects/c011ca03e43b4e53ac077f815a843e3a/users/a652da336e494c08921ce49bebce46fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.539257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.543932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.555910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:39.573502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c011ca03e43b4e53ac077f815a843e3a, user_id=a652da336e494c08921ce49bebce46fc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:39.576149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:39.582393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7b40a-0972-442b-ba6b-f3e116ae7b16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:39.613245 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2557/5046] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:39 2022] PUT /identity/v3/projects/c011ca03e43b4e53ac077f815a843e3a/users/a652da336e494c08921ce49bebce46fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:39.628627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.629123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.629559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.680393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa4b7f-17a1-466f-a80d-406472e2dadc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:39.686041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] MFA Rules not processed for user `a652da336e494c08921ce49bebce46fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:39.689974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:39.857612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:39.858267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:39.858579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:39.858860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:39.905000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2558/5047] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:39.917276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:39.920013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.074832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fcda9-e2ae-4efb-8f7c-8d32f7a9f477 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:40.081595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:40.082222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:40.083294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:40.083456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:40.111032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:40.112546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:40.116389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:40.120857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9e5a8-a97b-4900-b41b-9f5ca96379a0 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:40.136026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2490/5048] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:40.136820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:40.138132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.145617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2559/5049] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:40.184996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90174646-b308-4dab-ba1d-f8cb4d59f7c8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:40.196015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:40.197311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:40.197311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:40.197311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] PATH_INFO: `/v3/users/a19401f560ba4b089a9a652f90e445ed` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:40.203252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19401f560ba4b089a9a652f90e445ed)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:40.211289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:40.308384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19401f560ba4b089a9a652f90e445ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:40.309944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19401f560ba4b089a9a652f90e445ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:40.322484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19401f560ba4b089a9a652f90e445ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:40.335349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19401f560ba4b089a9a652f90e445ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:40.345197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19401f560ba4b089a9a652f90e445ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:40.365632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:40.406019 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2491/5050] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:40 2022] DELETE /identity/v3/users/a19401f560ba4b089a9a652f90e445ed => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:40.409033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:40.409396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:40.409748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:40.452663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:40.455380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.595364 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2560/5051] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:40.622659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:40.625026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.697727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:40.701438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.736301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:40.755637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b943187-4bf9-47e4-98c7-a199e0cd7bdc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:40.760042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:40.761491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:40.761870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:40.762291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:40.774312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:40.775952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:40.797239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2492/5052] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:40.915416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a225c-ee75-493b-a962-f0cdfceda8a4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:40.929198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:40.930663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:40.930957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:40.931230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:40.940052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:40.966429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:40.968348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:41.004177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2561/5053] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:42.160867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:42.162862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.184092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255337f-f047-49e2-8f12-7c2b60a7d326 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:42.200782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:42.201746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.202180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.202633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] PATH_INFO: `/v3/projects/3dc1d2f5cb904a11a86f9d691ea3df3a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.204020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc1d2f5cb904a11a86f9d691ea3df3a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:42.206122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:42.322994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:42.324699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.360100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b441b-7717-4df2-893b-c7dadfffb4dc service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:42.361417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc1d2f5cb904a11a86f9d691ea3df3a', 'request_id': 'req-2cffba2a-0b08-4258-907b-bea71dd1a626'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.373225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc1d2f5cb904a11a86f9d691ea3df3a', 'request_id': 'req-2cffba2a-0b08-4258-907b-bea71dd1a626'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.402758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc1d2f5cb904a11a86f9d691ea3df3a', 'request_id': 'req-2cffba2a-0b08-4258-907b-bea71dd1a626'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.440502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] The token cache is being invalidate because project 3dc1d2f5cb904a11a86f9d691ea3df3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:42.440806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.446381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.455471 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2493/5054] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:42 2022] DELETE /identity/v3/projects/3dc1d2f5cb904a11a86f9d691ea3df3a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:42.456517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:42.461381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.535462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:42.536716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.538763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.539435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] PATH_INFO: `/v3/users/a652da336e494c08921ce49bebce46fc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.541743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a652da336e494c08921ce49bebce46fc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:42.543955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:42.605205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffba2a-0b08-4258-907b-bea71dd1a626 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:42.616241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:42.618129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.618633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.619115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] PATH_INFO: `/v3/projects/2390e824cdda4bf98f61da52dcd574e7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.635628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2390e824cdda4bf98f61da52dcd574e7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:42.635628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:42.665347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a652da336e494c08921ce49bebce46fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.666073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a652da336e494c08921ce49bebce46fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.683844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a652da336e494c08921ce49bebce46fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.707391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:42.713257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a652da336e494c08921ce49bebce46fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.714238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a652da336e494c08921ce49bebce46fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.740561 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2562/5055] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:42 2022] DELETE /identity/v3/users/a652da336e494c08921ce49bebce46fc => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:42.742165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:42.745005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.765141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2390e824cdda4bf98f61da52dcd574e7', 'request_id': 'req-697b929a-ff7b-4b11-9e88-b1006f08a6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.775807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:42.779769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2390e824cdda4bf98f61da52dcd574e7', 'request_id': 'req-697b929a-ff7b-4b11-9e88-b1006f08a6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.804838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2390e824cdda4bf98f61da52dcd574e7', 'request_id': 'req-697b929a-ff7b-4b11-9e88-b1006f08a6d4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.819318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] The token cache is being invalidate because project 2390e824cdda4bf98f61da52dcd574e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:42.820638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.821361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-697b929a-ff7b-4b11-9e88-b1006f08a6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:42.824244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2494/5056] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:42 2022] DELETE /identity/v3/projects/2390e824cdda4bf98f61da52dcd574e7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:42.826335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.826921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.827261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.853504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:42.855274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.907035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb81389-7708-45f2-9e84-7b546e36ae94 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:42.920472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:42.921388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.921670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.922238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] PATH_INFO: `/v3/users/081e15c6bdd54c02888aa662ee032b2d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.923132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081e15c6bdd54c02888aa662ee032b2d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:42.930194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:42.944508 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2495/5057] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:42.945611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:42.947871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:42.970935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5774c1f-a5ca-4cb0-b91e-52d58bf92915 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:42.977723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:42.978454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:42.978933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:42.979287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] PATH_INFO: `/v3/users/b2ea0b9151df4b6cbb53f3d07b2c1922` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:42.980540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ea0b9151df4b6cbb53f3d07b2c1922)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:42.983600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:43.039338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081e15c6bdd54c02888aa662ee032b2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.039687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081e15c6bdd54c02888aa662ee032b2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.060548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081e15c6bdd54c02888aa662ee032b2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.070065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ea0b9151df4b6cbb53f3d07b2c1922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.070993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ea0b9151df4b6cbb53f3d07b2c1922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.084764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ea0b9151df4b6cbb53f3d07b2c1922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.088260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081e15c6bdd54c02888aa662ee032b2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.089750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081e15c6bdd54c02888aa662ee032b2d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.105601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ea0b9151df4b6cbb53f3d07b2c1922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.108575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ea0b9151df4b6cbb53f3d07b2c1922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:43.129733 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2563/5058] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:42 2022] DELETE /identity/v3/users/081e15c6bdd54c02888aa662ee032b2d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:43.131079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:43.134684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.161609 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2496/5059] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:42 2022] DELETE /identity/v3/users/b2ea0b9151df4b6cbb53f3d07b2c1922 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:43.169748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:43.172890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.237797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:43.299336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005450-a57e-48c8-a8eb-816ef1d66818 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:43.331548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee102f-d77c-47b6-93ab-532fefe100fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:43.341978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:43.342635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:43.342920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:43.343196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:43.347494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.369670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:43.434924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:43.438144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:43.438144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:43.438144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:43.441147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.514351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:43.517213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:43.571456 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2497/5060] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:43.703836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:43.708987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:43.723148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:43.725526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.739410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2564/5061] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:43.749915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34f013-331a-47c5-93ab-d80c6e02b838 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:43.762861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:43.763119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:43.764100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:43.764100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:43.775288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:43.780401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.793714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25c536-b2f2-4099-be30-60b3f0760a2c service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:43.795554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:43.797298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:43.808763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:43.809339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:43.809618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:43.809889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:43.811509 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2498/5062] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:43.817849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:43.847281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:43.848908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:43.861710 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2565/5063] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:10:44.215590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:44.229586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a8273-1cd3-4a9f-9212-4b8f883860df service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:44.245198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:44.245823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:44.246104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:44.246382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:44.255861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:44.283213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:44.285062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:44.320295 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2499/5064] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:44.714888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:44.718509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:44.923970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3956dc0-37a6-4c6b-9b83-5b8ac2898463 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:44.933489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:44.935239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:44.955560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:44.958945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:44.959261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:44.959918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:44.965269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:44.974375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf8cf0-104b-4e4f-b973-0289c44f6ab1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:44.985427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:44.986068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:44.986343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:44.986650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] PATH_INFO: `/v3/projects/c011ca03e43b4e53ac077f815a843e3a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:44.987831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c011ca03e43b4e53ac077f815a843e3a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:44.989803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:44.999860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:45.004183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:45.034980 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2566/5065] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:45.036712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:45.045224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.068892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0948a5a-30a0-4fbf-bfdf-224e7a91d30b service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:45.100770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c011ca03e43b4e53ac077f815a843e3a', 'request_id': 'req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.113127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c011ca03e43b4e53ac077f815a843e3a', 'request_id': 'req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.131176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c011ca03e43b4e53ac077f815a843e3a', 'request_id': 'req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.153727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] The token cache is being invalidate because project c011ca03e43b4e53ac077f815a843e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:45.154078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.154952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:45.156005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.156174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0112cf11-6b0a-4ae0-b340-ac3b33cf515d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.156463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.156831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] PATH_INFO: `/v3/projects/009521c189ab4b0ca2b073960b7ab359` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.158240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009521c189ab4b0ca2b073960b7ab359)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:45.160021 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2500/5066] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:44 2022] DELETE /identity/v3/projects/c011ca03e43b4e53ac077f815a843e3a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:45.161110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:45.166410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:45.187389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515100-bbab-4e42-a2b0-17880dde78e6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.187748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515100-bbab-4e42-a2b0-17880dde78e6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.187941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515100-bbab-4e42-a2b0-17880dde78e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.217483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83515100-bbab-4e42-a2b0-17880dde78e6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:45.221373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83515100-bbab-4e42-a2b0-17880dde78e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.241135 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2501/5067] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:45.243744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.244501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.244984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.271801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:45.274100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.275190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009521c189ab4b0ca2b073960b7ab359', 'request_id': 'req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.286471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009521c189ab4b0ca2b073960b7ab359', 'request_id': 'req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.297577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009521c189ab4b0ca2b073960b7ab359', 'request_id': 'req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.307239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:45.317087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] The token cache is being invalidate because project 009521c189ab4b0ca2b073960b7ab359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:45.318111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.319094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cc605b90-c409-4d4b-bab8-89dd0d56a3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:45.353085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2567/5068] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:45 2022] DELETE /identity/v3/projects/009521c189ab4b0ca2b073960b7ab359 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:45.354893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc646d-edbc-41ec-91e1-ec1f6895453d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.358278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc646d-edbc-41ec-91e1-ec1f6895453d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.358373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc646d-edbc-41ec-91e1-ec1f6895453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.417401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e8fc646d-edbc-41ec-91e1-ec1f6895453d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:45.432030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e8fc646d-edbc-41ec-91e1-ec1f6895453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.452948 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2502/5069] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:45.457587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:45.473163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7f48f-f754-4f95-8400-bf1d429039f7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:45.479517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:45.480409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.480844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.481215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.482077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:45.482476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:45.484527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:45.513089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2503/5070] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:45.517401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:45.538102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbacd2-0e62-4291-9597-fbb0a3718976 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:45.551697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:45.554606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.555547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.557599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.558640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:45.565388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:45.672992 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2504/5071] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:45.677739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:45.709479 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2568/5072] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:45.717881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.719142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.720296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.770147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:45.773034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.935682 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2569/5073] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:45.947665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:45.949722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094b60f-16b2-4bec-8cbe-49d0d5ea20a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:45.953351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:45.977289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:45.978025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:45.978309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:45.978643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:45.980850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:45.982898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.008418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef1bfa-6d9a-47f8-92eb-3e7091c9571c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.027227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.030058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.041044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.042649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] PATH_INFO: `/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.045486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdab24010d9f4e5b8f62a77a0ccfd999)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.054688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.093481 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2505/5074] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:45 2022] POST /identity/v3/users => generated 288 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:46.098338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.176327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eae20-6a3a-4dd5-9b21-db0d83bf0ab5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.204994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.205994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.206481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.207013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.207870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:46.208380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.210614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.245424 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2506/5075] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:46.246064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:46.265020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.297317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdab24010d9f4e5b8f62a77a0ccfd999', 'request_id': 'req-c9046728-b1e4-415d-92b2-8bb72733e1e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:46.312986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdab24010d9f4e5b8f62a77a0ccfd999', 'request_id': 'req-c9046728-b1e4-415d-92b2-8bb72733e1e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:46.332395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2966f-0aeb-4ee1-9cd2-9d97ea31ec3b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.342183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdab24010d9f4e5b8f62a77a0ccfd999', 'request_id': 'req-c9046728-b1e4-415d-92b2-8bb72733e1e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:46.383144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] The token cache is being invalidate because project fdab24010d9f4e5b8f62a77a0ccfd999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:46.383690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:46.386013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:46.392743 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2570/5076] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:45 2022] DELETE /identity/v3/projects/fdab24010d9f4e5b8f62a77a0ccfd999 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:46.393902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.406412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:46.444489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:46.486447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.487896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.488183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.488453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.490807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.495738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c83d9-bcea-4a53-b707-e233fcd00884 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.510146 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2507/5077] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:46.511824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.513002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.513002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.545407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:46.546806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:46.563700 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2508/5078] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:46.565307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.567387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:46.570111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046728-b1e4-415d-92b2-8bb72733e1e3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.585973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.589812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.590307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.590809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.591827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.597069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cbe39-bfbd-4e38-83e5-b3cb62536c47 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.603292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.604755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.605059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.605345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.605992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.607792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b655ff6-eca2-4112-a8ca-498e906b63b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.611867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.633240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2509/5079] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:46.638734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.639671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.639671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.692464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:46.693884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:46.702669 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2571/5080] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/projects => generated 397 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:46.703517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.722861 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2510/5081] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:46.724539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.738769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e98a4-e70b-4e90-9d47-eeabc33df38c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.793045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed6c8f-d20c-464d-b722-59b646e9960b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.797940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.798640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.798923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.799197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] PATH_INFO: `/v3/projects/1da1c85c3a394c4baa75e31ed90a6bae/users/f9343f7112da4c48aad132a7f648ada0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.816455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.817317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.817598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.817866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.818182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da1c85c3a394c4baa75e31ed90a6bae, user_id=f9343f7112da4c48aad132a7f648ada0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.818486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.820722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5349d-0682-4ce7-a835-e97545e85079 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.821126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.836686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2572/5082] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:46 2022] PUT /identity/v3/projects/1da1c85c3a394c4baa75e31ed90a6bae/users/f9343f7112da4c48aad132a7f648ada0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:46.844257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.847977 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2511/5083] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:46.849684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.850357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.851444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.880766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:46.886434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] MFA Rules not processed for user `f9343f7112da4c48aad132a7f648ada0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:46.889127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:46.936118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25ce32-4938-4eb6-b598-daada6311505 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.956747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:46.957525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:46.958708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:46.958984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:46.960041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:46.960192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:46.962473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:46.980044 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2573/5084] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:46.981009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:46.994560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f8a6-804b-4157-89e3-618cabd50157 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:46.997311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2512/5085] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:46.998354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.003573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.004203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.004483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.004754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.005717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.007435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.013571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af73ba1-b556-48e7-b81a-f95598fdbb31 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.018518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.019218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.019569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.019916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.020651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.024124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.040961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2574/5086] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:47.043231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.066532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439c2a-631c-4b31-bd20-0a61ee2e298c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.074622 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2513/5087] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:46 2022] POST /identity/v3/projects => generated 369 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:47.075634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.079196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.079845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.080120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.080387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.081153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.085082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.111982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:47.127150 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2575/5088] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:47 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:47.129592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.160450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24b39-0cdc-481c-8f23-5851ded80581 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.267955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a348-394a-424e-a581-8bf3b89068ae admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.276596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.278177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.278753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.279522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.281485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.285507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.312554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.314381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.315357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.316685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] PATH_INFO: `/v3/projects/9218229682264b38bf2a838c76395159/users/230438e3c3784c329be318a5821aad3a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.322644 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2576/5089] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:47 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:47.323549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.346900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a778-b21a-4aea-9b80-0290850098e4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.349281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9218229682264b38bf2a838c76395159, user_id=230438e3c3784c329be318a5821aad3a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.353598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.354308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.354692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.355040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.355758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.357341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.358010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.381927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2514/5090] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:47 2022] PUT /identity/v3/projects/9218229682264b38bf2a838c76395159/users/230438e3c3784c329be318a5821aad3a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:47.384705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.417900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2577/5091] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:47 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:47.418960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.455368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c37b9-34a5-4b04-ae73-6144bd35e85e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.487632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f887e19-c132-4f6d-b8be-15bf3801f656 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.494561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.495577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.496106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.496617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.497349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.499113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.520628 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2515/5092] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:47.521671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.536862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d012063-b066-4866-be4c-f8032906f9fe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.542639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.543272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.545394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.545903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.546352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.548217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.564433 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2516/5093] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:47.565847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.586186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08b637-51f8-4507-9256-7f2a7fd2529f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.591138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.591846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.592788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.593116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.593766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.595589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.639490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:47.645630 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2517/5094] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:47.649136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.662804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.663883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.664268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.664662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.668374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.671236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.674236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a06aa-3340-4fde-82a7-7c7a5db12294 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.679709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.680767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.681106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.681387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] PATH_INFO: `/v3/projects/20425031dd4a4f40b8e6b1b75af25fff/users/516a7da2f28d442385162894922e6532/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.713876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20425031dd4a4f40b8e6b1b75af25fff, user_id=516a7da2f28d442385162894922e6532, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.714808 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2578/5095] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:47.716634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.717091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.736538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5603fc-14df-4652-bde2-89f3fa777cdb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.750592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2518/5096] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:47 2022] PUT /identity/v3/projects/20425031dd4a4f40b8e6b1b75af25fff/users/516a7da2f28d442385162894922e6532/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:47.750592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.758542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:47.870720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.871453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.872321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.872607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] PATH_INFO: `/v3/projects/8d0cb5ff1eb0440abace7fc7d9bb16ef/users/a8f280ab42484b9b95b64f1886088816/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.892044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf338b6-7c38-4c8d-8a80-ea5a5d8f7675 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:47.899121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:47.900151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.900453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.902632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] PATH_INFO: `/v3/projects/9218229682264b38bf2a838c76395159/users/230438e3c3784c329be318a5821aad3a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:47.904474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9218229682264b38bf2a838c76395159, user_id=230438e3c3784c329be318a5821aad3a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.907828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.927448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0cb5ff1eb0440abace7fc7d9bb16ef, user_id=a8f280ab42484b9b95b64f1886088816, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:47.928581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2519/5097] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:47 2022] PUT /identity/v3/projects/9218229682264b38bf2a838c76395159/users/230438e3c3784c329be318a5821aad3a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:47.931595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:47.935569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a4341-e85d-4b54-a60c-5c2f1c038a00 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:47.958698 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2579/5098] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:47 2022] PUT /identity/v3/projects/8d0cb5ff1eb0440abace7fc7d9bb16ef/users/a8f280ab42484b9b95b64f1886088816/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:47.961625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cc54-9bf2-4c1e-a1e6-1df140c6ab93 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:47.962123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cc54-9bf2-4c1e-a1e6-1df140c6ab93 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:47.962675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45cc54-9bf2-4c1e-a1e6-1df140c6ab93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.008597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5b45cc54-9bf2-4c1e-a1e6-1df140c6ab93 None None] MFA Rules not processed for user `230438e3c3784c329be318a5821aad3a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:48.015783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5b45cc54-9bf2-4c1e-a1e6-1df140c6ab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:48.053330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3e577-b3b6-4dd6-9d68-36165786af64 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.151631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2580/5099] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:47 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:48.153741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.154049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.154325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.175196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.175827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.176111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.176385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.177038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.180048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.204402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2520/5100] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:48.205313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.215016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] MFA Rules not processed for user `a8f280ab42484b9b95b64f1886088816`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:48.217187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2f38f-7e07-492b-b89d-e1419f69ae38 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.217610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:48.227631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.228474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.228760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.229108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.229773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.232256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.291354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2521/5101] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/projects => generated 397 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:48.292773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.312559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:48.322007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:48.338194 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2581/5102] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:48.339151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.389765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb563a66-c0d1-455c-b44c-1b43f7a45960 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.394110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace848c-62f0-415a-a373-a88a657c7ddd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.403124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.403855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.404156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.404432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.404537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.405150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.405260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.405427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.405962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] PATH_INFO: `/v3/projects/20425031dd4a4f40b8e6b1b75af25fff/users/516a7da2f28d442385162894922e6532/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.408032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=20425031dd4a4f40b8e6b1b75af25fff, user_id=516a7da2f28d442385162894922e6532, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.409425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d892-5fcc-447f-b6fb-b7634af39384 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.410751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.434131 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2522/5103] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:48 2022] PUT /identity/v3/projects/20425031dd4a4f40b8e6b1b75af25fff/users/516a7da2f28d442385162894922e6532/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:48.435392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.460264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2582/5104] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:48.465420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.467130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.467277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.507296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] MFA Rules not processed for user `516a7da2f28d442385162894922e6532`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:48.508854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:48.571719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d535280-d2dd-47e8-a7a6-29ba7d555dfc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.578617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.579428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.579814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.580285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.581165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.583222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.601803 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2583/5105] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:48.602750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.616303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbcd57b-cf3e-47d7-ba71-577d81dc00e7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.622738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.623365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.623988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.623988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.624535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.626333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.641792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2584/5106] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:48.642792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.657026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2523/5107] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/projects => generated 369 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:48.660504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e79df7-f9a2-4994-b77e-3815d55afcaf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.661607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.735012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.735640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.735930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.736205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.736847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.738912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.765445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89a445-2d4a-4b5f-a258-8ed86acb1603 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.773905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.774689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.775451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.775835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.776703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.779399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.783558 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2585/5108] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/projects => generated 351 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:48.785067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.802473 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2524/5109] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:48.802473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.831756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ffb93-bd47-4d27-a460-c3d862502357 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.837643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:48.876637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edd338-ee8c-4361-b0ba-29b53eaf9d83 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.883267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.884103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.884386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.884656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.885300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.887487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:48.887796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.920710 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2586/5110] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:48.922105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:48.935708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a1ef-6ed7-4395-ba50-27dece1b8250 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:48.943326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:48.943922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:48.944303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:48.944834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] PATH_INFO: `/v3/projects/9facbd9b3a69461e8a4aeb7b95be9297/users/affd65bfbfbd485280606d15f1199fa4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:48.973623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9facbd9b3a69461e8a4aeb7b95be9297, user_id=affd65bfbfbd485280606d15f1199fa4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:48.994432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:48.995545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.001124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.001428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.001833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.002598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.012963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647374cd-f2fa-40af-bb60-6126aad6901d tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.034802 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2587/5111] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:48 2022] PUT /identity/v3/projects/9facbd9b3a69461e8a4aeb7b95be9297/users/affd65bfbfbd485280606d15f1199fa4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:49.039270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.090865 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2525/5112] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:48 2022] POST /identity/v3/projects => generated 350 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:49.093033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.093862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.094298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.136525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] MFA Rules not processed for user `affd65bfbfbd485280606d15f1199fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:49.138543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:49.138778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664e187-97c2-42ae-a984-4d649ded9870 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.268018 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2526/5113] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:49.269622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.299332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.300474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.300904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.301328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.306112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.308989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.332746 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2588/5114] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:49.337242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.342416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:49.358844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7488e5-3c7c-4e26-8263-35eb4a3ebd1f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.368170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.368917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.369259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.369607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/users/f9343f7112da4c48aad132a7f648ada0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.371605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:get_user(user_id=f9343f7112da4c48aad132a7f648ada0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.374732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.379881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2527/5115] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:10:49 2022] GET /identity/v3/users/f9343f7112da4c48aad132a7f648ada0 => generated 288 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:49.380891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.395863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603078a-05a1-4a95-9ca5-14b085620c77 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.402317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.403014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.403290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.403562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] PATH_INFO: `/v3/projects/a77a8cca971441e1adc221f24d95799e/users/f6d9cb502b66409e803a84ae56a2754c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.428543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77a8cca971441e1adc221f24d95799e, user_id=f6d9cb502b66409e803a84ae56a2754c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.434444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.462918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:49.496368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95d778-4787-4e23-aa49-7569962d233d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.504423 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2528/5116] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:49 2022] PUT /identity/v3/projects/a77a8cca971441e1adc221f24d95799e/users/f6d9cb502b66409e803a84ae56a2754c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:49.505619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.559670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:49.603105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae6a1b-0532-480e-8208-3c446db50858 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.609349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.610142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.610511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.611236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.611967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.614011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.616146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.617355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.617703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.618091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.680685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2529/5117] 10.222.0.52 () {60 vars in 1162 bytes} [Sat Sep 24 21:10:49 2022] POST /identity/v3/credentials => generated 430 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:49.682000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.783525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.784921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.825975 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2589/5118] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:49.825975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.843272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:49.848722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70601368-dbe8-4bf5-ae53-9b04c9272d72 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.868984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.869607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.870069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.870353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.871169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.879638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.903730 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2530/5119] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:49.905064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:49.932747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cf24f-0484-453a-b8bd-8e718f34aa1b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.943714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.944362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.946539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.946871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.955207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.957132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.957696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703020ea-ac56-42c1-8f6f-29e3379da414 service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:49.962064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:49.964642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:49.965346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:49.965630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:49.966893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:49.968058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:49.969587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:49.997300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2531/5120] 10.222.0.52 () {60 vars in 1358 bytes} [Sat Sep 24 21:10:49 2022] PATCH /identity/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777 => generated 436 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:49.998577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.012590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8796323-f89d-416d-b2fa-e23d8ebd37b7 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.018566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.020720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.023503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.024494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.024974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.025617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] PATH_INFO: `/v3/projects/a77a8cca971441e1adc221f24d95799e/users/f6d9cb502b66409e803a84ae56a2754c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.028889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77a8cca971441e1adc221f24d95799e, user_id=f6d9cb502b66409e803a84ae56a2754c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.033331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d74dc9-a1f7-4787-aebd-27a9392b22a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.056868 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2590/5121] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:50.058050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.070001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2532/5122] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:49 2022] PUT /identity/v3/projects/a77a8cca971441e1adc221f24d95799e/users/f6d9cb502b66409e803a84ae56a2754c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:50.071998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.072292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.072573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.074518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb882-545e-4a5a-b35e-48251d86ad0f service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.128048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:50.181222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] MFA Rules not processed for user `f6d9cb502b66409e803a84ae56a2754c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:50.182546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:50.228054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.231556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.231841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.232157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.239935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.246245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.253103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2591/5123] 10.222.0.52 () {58 vars in 1335 bytes} [Sat Sep 24 21:10:50 2022] GET /identity/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777 => generated 436 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:50.258952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.271940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2675c4-df87-4402-9a39-d2e440eef23c tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.307420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.309972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.311111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.313479 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2533/5124] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:50.313685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.316852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.318638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.325410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.342818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09243a0-024a-4950-bd40-4bbbc5e9efb5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.354687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.357283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.357569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.359172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.359835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.365531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.379702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2592/5125] 10.222.0.52 () {58 vars in 1338 bytes} [Sat Sep 24 21:10:50 2022] DELETE /identity/v3/credentials/4b1d4c8e0762cee86825bfc73c4765d0d6ac4b743096726b851a644209465777 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:50.388664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.408198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:50.422640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70278276-b9e0-4ecd-8917-8d171582fbc4 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.433778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2534/5126] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/projects => generated 328 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:50.435914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.436683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.436973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.437257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.439548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.440991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.444054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.483225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2593/5127] 10.222.0.52 () {60 vars in 1162 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/credentials => generated 430 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:50.484205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.496471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f7db-6f13-4f78-bb54-b5720157e3f3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.565853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54b888-8d4c-401c-bc40-92de05c8a76d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.573485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.574362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.574685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.575158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.575886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.578240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.630164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2535/5128] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/users => generated 416 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:50.637325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.667350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b683b-aa62-4388-88b9-4f71ef8e5f1c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.690143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.690792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.691071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.691347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.692016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.696465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.715571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.717889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.722531 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2536/5129] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/roles => generated 243 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:50.722712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.722712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.722712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.724566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.733154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.754261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482392-ef3e-4f3f-bc73-8d6c698858f1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.762224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.762888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.763442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.763631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.764898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.767409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.781435 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2594/5130] 10.222.0.52 () {60 vars in 1162 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/credentials => generated 430 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:50.786289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.801436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2537/5131] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:50 2022] POST /identity/v3/roles => generated 246 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:50.805729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.810797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e662b1-84a8-40dc-81e3-396de802cb9e tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.840453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.841822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.842697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.842697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.846235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.852524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.876782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.881255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.885053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.889498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:50.899144 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2595/5132] 10.222.0.52 () {58 vars in 1140 bytes} [Sat Sep 24 21:10:50 2022] GET /identity/v3/credentials => generated 945 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:50.900097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:50.932201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0227af-ce73-493a-af4d-ad723101e438 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:50.944921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:50.947060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:50.947347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:50.947623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64/users/a3145da001ac450699991c92f5b6b51e/roles/a441f531ed11488883850ba02829f3ed` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:50.992413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=df2aaa9da68a401b8a05cc8a0fe7ff64, user_id=a3145da001ac450699991c92f5b6b51e, role_id=a441f531ed11488883850ba02829f3ed)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:50.994170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815e11-fe2b-45f4-9060-8be7606e7ea5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.004638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.012152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.014037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.014321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.015904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] PATH_INFO: `/v3/users/230438e3c3784c329be318a5821aad3a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.018057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=230438e3c3784c329be318a5821aad3a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:51.023455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.041933 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2596/5133] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:50 2022] PUT /identity/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64/users/a3145da001ac450699991c92f5b6b51e/roles/a441f531ed11488883850ba02829f3ed => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:51.044828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.163204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230438e3c3784c329be318a5821aad3a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:51.164171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230438e3c3784c329be318a5821aad3a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:51.181625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230438e3c3784c329be318a5821aad3a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:51.191145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:51.196961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230438e3c3784c329be318a5821aad3a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:51.199261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230438e3c3784c329be318a5821aad3a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:51.213162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a0fe4-d298-4850-abce-7cafcbce3afd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.222679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.223695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.224394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.224891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials/f50b72a4d328c819ebe498eeb58752aad542e7ab84d6d08ae11fb9543b4549eb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.230083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=f50b72a4d328c819ebe498eeb58752aad542e7ab84d6d08ae11fb9543b4549eb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:51.233709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56e46-fabc-40f3-9e89-03952b13536a tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.244655 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2538/5134] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:50 2022] DELETE /identity/v3/users/230438e3c3784c329be318a5821aad3a => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:51.247501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.250247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.278640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:51.299534 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2597/5135] 10.222.0.52 () {58 vars in 1338 bytes} [Sat Sep 24 21:10:51 2022] DELETE /identity/v3/credentials/f50b72a4d328c819ebe498eeb58752aad542e7ab84d6d08ae11fb9543b4549eb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:51.301290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.301817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.302551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.376479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:51.379044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.442593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d9f36-d926-44c5-b2c2-2d82800f8b4c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.457037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.458243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.459112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.459787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64/users/a3145da001ac450699991c92f5b6b51e/roles/ca2cd4e8e4e9410bae7b00f3a0460d6d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.462380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=df2aaa9da68a401b8a05cc8a0fe7ff64, user_id=a3145da001ac450699991c92f5b6b51e, role_id=ca2cd4e8e4e9410bae7b00f3a0460d6d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:51.465523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae149d-a066-4f30-8157-9ac8f7f59cc9 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.489103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2539/5136] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:51 2022] PUT /identity/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64/users/a3145da001ac450699991c92f5b6b51e/roles/ca2cd4e8e4e9410bae7b00f3a0460d6d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:51.490981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.491937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.492255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.493254 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2598/5137] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:51.496144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.497866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.519422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:51.524367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.631723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaec653-e5c2-49fe-bc12-c55b0172fb8c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.682406 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2540/5138] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:51.682406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.718362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.719557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.720098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.720627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/credentials/fa391498dfd450a8ad431eac5840b9487ecce0bea52cdbf5f8629c265c692482` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.722933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=fa391498dfd450a8ad431eac5840b9487ecce0bea52cdbf5f8629c265c692482)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:51.725725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.753514 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2599/5139] 10.222.0.52 () {58 vars in 1338 bytes} [Sat Sep 24 21:10:51 2022] DELETE /identity/v3/credentials/fa391498dfd450a8ad431eac5840b9487ecce0bea52cdbf5f8629c265c692482 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:51.758696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.761028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.779816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:51.837724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef510c9-88a7-4d3d-9604-567cc1f49769 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.843179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.843907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.844036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.844251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.844919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:51.849558 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:51.849558 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:10:51.849558 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:10:51.850372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:51.947363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2541/5140] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:51 2022] GET /identity/v3/users => generated 8633 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:51.947363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:51.947363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:51.951777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4b33c-4ab6-474d-abe9-19b512350aeb tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:51.961487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:51.962344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:51.962733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:51.963355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:51.978760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:52.057301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02eb6-ac37-46af-80ad-9a88d881f363 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:52.062460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:52.063223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:52.063610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:52.064140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:52.078710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:52.088175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:52.102354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2542/5141] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:10:52.103426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:52.104955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:52.108493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c2c81-3e3b-4f8b-a6df-a599ced28bf7 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:52.114445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdaa5c-f133-49ad-bcba-c69c380d0932 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:52.122450 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2600/5142] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:10:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:10:52.125211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0047d16-44e2-46a1-a944-e5e549c471b8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:52.125597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0047d16-44e2-46a1-a944-e5e549c471b8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:52.126473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0047d16-44e2-46a1-a944-e5e549c471b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:52.142196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:52.143215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:52.143806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:52.144238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/projects/aa4d265165bf4370a79aef2f6e9a093c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:52.145954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aa4d265165bf4370a79aef2f6e9a093c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:52.150006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:52.187276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0047d16-44e2-46a1-a944-e5e549c471b8 None None] MFA Rules not processed for user `a3145da001ac450699991c92f5b6b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:52.189093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0047d16-44e2-46a1-a944-e5e549c471b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:52.265590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4d265165bf4370a79aef2f6e9a093c', 'request_id': 'req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.277846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4d265165bf4370a79aef2f6e9a093c', 'request_id': 'req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.304369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4d265165bf4370a79aef2f6e9a093c', 'request_id': 'req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.325577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] The token cache is being invalidate because project aa4d265165bf4370a79aef2f6e9a093c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:52.326335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.327302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.336511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2543/5143] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:52 2022] DELETE /identity/v3/projects/aa4d265165bf4370a79aef2f6e9a093c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:52.337821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:52.347779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:52.504568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d9145-cc2a-4622-a00e-e92b81c0cab3 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:52.521870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516a7da2f28d442385162894922e6532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20425031dd4a4f40b8e6b1b75af25fff', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:52.523095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:52.523543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:52.523965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] PATH_INFO: `/v3/projects/cf0635f52362445eb027178cd33d62be` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:52.525913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cf0635f52362445eb027178cd33d62be)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:52.528236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:52.650208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:52.674884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0635f52362445eb027178cd33d62be', 'request_id': 'req-db95598e-53e5-41fd-bcb2-2aafa44fa460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.692411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0635f52362445eb027178cd33d62be', 'request_id': 'req-db95598e-53e5-41fd-bcb2-2aafa44fa460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.708261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0635f52362445eb027178cd33d62be', 'request_id': 'req-db95598e-53e5-41fd-bcb2-2aafa44fa460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.736934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] The token cache is being invalidate because project cf0635f52362445eb027178cd33d62be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:52.738832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.740197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:52.744975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2544/5144] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:52 2022] DELETE /identity/v3/projects/cf0635f52362445eb027178cd33d62be => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:52.749481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:52.753965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:52.847242 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2601/5145] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 724 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:52.852195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:52.852687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:52.853115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:52.910679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] MFA Rules not processed for user `a3145da001ac450699991c92f5b6b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:52.915068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:52.946401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:53.005302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95598e-53e5-41fd-bcb2-2aafa44fa460 tempest-CredentialsTestJSON-805250945 tempest-CredentialsTestJSON-805250945-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:53.030434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:53.031637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:53.031921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:53.032189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] PATH_INFO: `/v3/users/f9343f7112da4c48aad132a7f648ada0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:53.035629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9343f7112da4c48aad132a7f648ada0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:53.045949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:53.103725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2602/5146] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:53.119550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:53.155448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9343f7112da4c48aad132a7f648ada0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:53.158653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a520116-16fd-4ba0-9626-05001bc608a1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:53.159744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9343f7112da4c48aad132a7f648ada0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:53.168730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3145da001ac450699991c92f5b6b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-245124519', 'tempest-NotDelegatedRole-926147288'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:53.169591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:53.170592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:53.170870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:53.173098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: Authorizing `identity:create_trust()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:53.176461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:53.186484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9343f7112da4c48aad132a7f648ada0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:53.199165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9343f7112da4c48aad132a7f648ada0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:53.200942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-45e4efa9-6436-4125-9ed5-691a99c6325f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9343f7112da4c48aad132a7f648ada0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:53.243084 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2545/5147] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:52 2022] DELETE /identity/v3/users/f9343f7112da4c48aad132a7f648ada0 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:53.247618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:53.248114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:53.248576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:53.294368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:53.298686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:53.401056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2603/5148] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:10:53 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:53.405073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:53.407746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:53.422866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:53.503383 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2546/5149] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:53.572801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:53.574422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:53.807814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a866bad-e9b6-4d37-bd15-0dbf3a3c0566 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:53.813787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:53.814416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:53.814713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:53.814983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:53.847776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:53.851146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8ef22-8947-4249-8d42-83cf5094a310 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:53.852298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:53.923928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2547/5150] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:10:53.924878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:53.929688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:53.963801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d51c43-c643-4fea-90ec-653fdfa0da99 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:53.979391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:53.979832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:53.980112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:53.980387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] PATH_INFO: `/v3/projects/9218229682264b38bf2a838c76395159` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:53.983225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9218229682264b38bf2a838c76395159)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:53.985194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:54.031910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:54.051222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:54.051982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:54.052379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:54.052770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:54.054145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:54.055462 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:54.055462 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:54.055462 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:54.056967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:54.077480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2604/5151] 10.222.0.52 () {58 vars in 1155 bytes} [Sat Sep 24 21:10:53 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 670 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:54.082071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:54.087864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:54.127583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9218229682264b38bf2a838c76395159', 'request_id': 'req-9f8ce1ab-ee05-4656-b23d-65b671e223d0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.170860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9218229682264b38bf2a838c76395159', 'request_id': 'req-9f8ce1ab-ee05-4656-b23d-65b671e223d0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.191069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9218229682264b38bf2a838c76395159', 'request_id': 'req-9f8ce1ab-ee05-4656-b23d-65b671e223d0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.229089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] The token cache is being invalidate because project 9218229682264b38bf2a838c76395159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:54.229912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.230980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.234169 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2548/5152] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:53 2022] DELETE /identity/v3/projects/9218229682264b38bf2a838c76395159 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:54.239193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:54.245839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:54.295489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73895a4c-608e-4e88-b12d-32cfda9628b6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:54.471971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ce1ab-ee05-4656-b23d-65b671e223d0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:54.481545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3145da001ac450699991c92f5b6b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-245124519', 'tempest-NotDelegatedRole-926147288'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:54.482389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:54.483452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:54.485300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] PATH_INFO: `/v3/OS-TRUST/trusts/f91885a7d6da4cba90ee645f810abb00` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:54.489040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:54.490134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:54.490997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:54.493562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] PATH_INFO: `/v3/users/affd65bfbfbd485280606d15f1199fa4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:54.499147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=affd65bfbfbd485280606d15f1199fa4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:54.504686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:54.511870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: Authorizing `identity:delete_trust(trust_id=f91885a7d6da4cba90ee645f810abb00)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:54.515940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:54.536529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:54.579576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'f91885a7d6da4cba90ee645f810abb00', 'request_id': 'req-1484b84c-ba3f-440d-9262-bf6f9a50972a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.594305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'f91885a7d6da4cba90ee645f810abb00', 'request_id': 'req-1484b84c-ba3f-440d-9262-bf6f9a50972a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.597086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'f91885a7d6da4cba90ee645f810abb00', 'request_id': 'req-1484b84c-ba3f-440d-9262-bf6f9a50972a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.613752 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2605/5153] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:10:54 2022] DELETE /identity/v3/OS-TRUST/trusts/f91885a7d6da4cba90ee645f810abb00 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:54.618093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:54.621784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:54.651635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'affd65bfbfbd485280606d15f1199fa4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.655164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'affd65bfbfbd485280606d15f1199fa4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.658414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484b84c-ba3f-440d-9262-bf6f9a50972a tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:54.675763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3145da001ac450699991c92f5b6b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-245124519', 'tempest-NotDelegatedRole-926147288'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:54.676748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:54.677035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:54.677415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] PATH_INFO: `/v3/OS-TRUST/trusts/f91885a7d6da4cba90ee645f810abb00` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:54.703297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'affd65bfbfbd485280606d15f1199fa4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Could not find trust: f91885a7d6da4cba90ee645f810abb00.: keystone.exception.TrustNotFound: Could not find trust: f91885a7d6da4cba90ee645f810abb00. Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 24 21:10:54.752783 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: f91885a7d6da4cba90ee645f810abb00. Sep 24 21:10:54.753972 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:10:54.758121 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2606/5154] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:10:54 2022] GET /identity/v3/OS-TRUST/trusts/f91885a7d6da4cba90ee645f810abb00 => generated 111 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:54.764601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'affd65bfbfbd485280606d15f1199fa4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.765864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:54.768424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:54.770217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'affd65bfbfbd485280606d15f1199fa4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:54.872403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2549/5155] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:54 2022] DELETE /identity/v3/users/affd65bfbfbd485280606d15f1199fa4 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:54.930267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef150f7-a9cd-40c0-b9f7-f30a9ea9dbb5 tempest-TrustsV3TestJSON-384318926 tempest-user-1735574606] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:54.966192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:55.105973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:55.106829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:55.107114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:55.107281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/ca2cd4e8e4e9410bae7b00f3a0460d6d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:55.110542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ca2cd4e8e4e9410bae7b00f3a0460d6d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:55.115947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:55.221013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role ca2cd4e8e4e9410bae7b00f3a0460d6d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:55.221363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.222970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.228384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2607/5156] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:54 2022] DELETE /identity/v3/roles/ca2cd4e8e4e9410bae7b00f3a0460d6d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:55.232302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:55.257988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daecdf2-f8fa-47c0-8359-1e77552b1e36 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:55.412417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:55.415070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:55.423183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:55.423908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/a441f531ed11488883850ba02829f3ed` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:55.427039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a441f531ed11488883850ba02829f3ed)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:55.431656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:55.523246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role a441f531ed11488883850ba02829f3ed has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:55.523802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.525259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.532976 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2550/5157] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:55 2022] DELETE /identity/v3/roles/a441f531ed11488883850ba02829f3ed => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:55.536260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:55.538054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:55.584880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c9fc3-4209-4657-9c50-df01320f6fa5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:55.783579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:55.786988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:55.787332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:55.787612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/a3145da001ac450699991c92f5b6b51e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:55.800047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a3145da001ac450699991c92f5b6b51e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:55.803121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:55.960379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3145da001ac450699991c92f5b6b51e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.962535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3145da001ac450699991c92f5b6b51e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:55.983998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3145da001ac450699991c92f5b6b51e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.013141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3145da001ac450699991c92f5b6b51e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.014711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3145da001ac450699991c92f5b6b51e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.048513 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2608/5158] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:55 2022] DELETE /identity/v3/users/a3145da001ac450699991c92f5b6b51e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:56.051676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.053374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:56.077285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c9b4-4ecc-4b5d-8a8d-096c11120fe2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:56.162319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:56.173304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:56.174170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:56.174844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:56.175678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:56.177119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=df2aaa9da68a401b8a05cc8a0fe7ff64)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:56.179364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:56.290117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'request_id': 'req-6dfc62c8-b9fe-4c30-b802-481c241e27b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.297511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'request_id': 'req-6dfc62c8-b9fe-4c30-b802-481c241e27b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.303970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.306810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:56.314390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:56.341487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2aaa9da68a401b8a05cc8a0fe7ff64', 'request_id': 'req-6dfc62c8-b9fe-4c30-b802-481c241e27b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.365907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project df2aaa9da68a401b8a05cc8a0fe7ff64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:56.366114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.369291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.377242 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2551/5159] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:56 2022] DELETE /identity/v3/projects/df2aaa9da68a401b8a05cc8a0fe7ff64 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:56.378587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.384896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:56.510575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210df7c-1581-4554-99ad-a100ef0e56c3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:56.533706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc62c8-b9fe-4c30-b802-481c241e27b5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:56.547370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:56.548080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:56.548368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:56.548642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:56.549293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:56.551615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:56.620104 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2552/5160] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:10:56 2022] POST /identity/v3/projects => generated 329 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:56.624757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.705981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:56.706820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:56.707354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:56.707893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] PATH_INFO: `/v3/projects/1da1c85c3a394c4baa75e31ed90a6bae` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:56.709594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da1c85c3a394c4baa75e31ed90a6bae)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:56.713064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:56.720612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccfd7f-0eac-43ff-857c-245179d892f7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:56.734184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:56.735279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:56.735564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:56.737300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:56.739770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:56.746052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:56.790622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:56.804714 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2553/5161] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:56 2022] POST /identity/v3/users => generated 416 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:56.806009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da1c85c3a394c4baa75e31ed90a6bae', 'request_id': 'req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.806653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.817635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da1c85c3a394c4baa75e31ed90a6bae', 'request_id': 'req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.828128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da1c85c3a394c4baa75e31ed90a6bae', 'request_id': 'req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.832497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1d8da-668d-42d3-9287-771117234676 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:56.869706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] The token cache is being invalidate because project 1da1c85c3a394c4baa75e31ed90a6bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:56.872963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.878623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:56.882276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2609/5162] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:56 2022] DELETE /identity/v3/projects/1da1c85c3a394c4baa75e31ed90a6bae => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:56.885768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.888392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:56.950762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:56.951736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:56.952269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:56.952703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:56.953729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:56.956776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:56.961164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:56.991964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2554/5163] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:56 2022] POST /identity/v3/roles => generated 244 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:56.993686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:56.996874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:57.010372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b9d7e-d6d3-445b-9fed-5284bcd868a4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:57.013468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df377a21-ab4f-4ca1-a463-ae34c9a026a7 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:57.020777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:57.021661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.023399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.023858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:57.025531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:57.028174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:57.042082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:57.043361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.046945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.047627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] PATH_INFO: `/v3/users/516a7da2f28d442385162894922e6532` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:57.052020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=516a7da2f28d442385162894922e6532)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:57.057260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2555/5164] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:10:56 2022] POST /identity/v3/roles => generated 247 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:57.063498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:57.064084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:57.101763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1116e2-660e-412c-9408-fbbef7369f71 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:57.124261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:57.125508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.126025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.126546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/64e1050175b74aad9a721ed6f5e2efcb/users/f2465e6ec8d94ee594e238d50fe29691/roles/db08f56f1ca64a419e11ea69e68238b7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:57.164700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=64e1050175b74aad9a721ed6f5e2efcb, user_id=f2465e6ec8d94ee594e238d50fe29691, role_id=db08f56f1ca64a419e11ea69e68238b7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:57.170239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:57.181975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516a7da2f28d442385162894922e6532'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:57.183746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516a7da2f28d442385162894922e6532'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:57.194988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516a7da2f28d442385162894922e6532'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:57.206342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2556/5165] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:57 2022] PUT /identity/v3/projects/64e1050175b74aad9a721ed6f5e2efcb/users/f2465e6ec8d94ee594e238d50fe29691/roles/db08f56f1ca64a419e11ea69e68238b7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:57.211440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:57.213036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516a7da2f28d442385162894922e6532'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:57.214254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516a7da2f28d442385162894922e6532'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:57.214692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:57.249465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2610/5166] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:56 2022] DELETE /identity/v3/users/516a7da2f28d442385162894922e6532 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:57.355851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db2859-ba94-443a-860f-1bd136c4b9a8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:57.480289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:57.483023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.484890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.485955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/64e1050175b74aad9a721ed6f5e2efcb/users/f2465e6ec8d94ee594e238d50fe29691/roles/d0d27654f2564f229b1c4ed97bd79229` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:57.492336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=64e1050175b74aad9a721ed6f5e2efcb, user_id=f2465e6ec8d94ee594e238d50fe29691, role_id=d0d27654f2564f229b1c4ed97bd79229)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:57.500265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f076b-4187-4447-bb1d-812a6c7802dc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:57.511398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:57.554067 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2557/5167] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:57 2022] PUT /identity/v3/projects/64e1050175b74aad9a721ed6f5e2efcb/users/f2465e6ec8d94ee594e238d50fe29691/roles/d0d27654f2564f229b1c4ed97bd79229 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:57.556042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:57.581451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44f095-1682-45f7-bbdf-f7ce21fe9b0b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:57.777237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:57.779635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.782085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.782624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:57.784225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:57.786816 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:10:57.786816 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:10:57.786816 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:10:57.790164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:57.897394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2611/5168] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:57 2022] GET /identity/v3/users => generated 7780 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:57.904029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff71d7-dcbf-45d5-a966-f37f80134f8e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:57.904029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff71d7-dcbf-45d5-a966-f37f80134f8e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:57.904340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff71d7-dcbf-45d5-a966-f37f80134f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.005174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d6ff71d7-dcbf-45d5-a966-f37f80134f8e None None] MFA Rules not processed for user `f2465e6ec8d94ee594e238d50fe29691`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:58.005174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d6ff71d7-dcbf-45d5-a966-f37f80134f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.049047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:58.051284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.196110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2558/5169] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:58.201270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:58.204031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.204749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.206320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.231128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473eaad-54e7-435b-ba0b-12fed8c271f5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:58.269480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:58.270354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.270808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.274354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] PATH_INFO: `/v3/projects/9facbd9b3a69461e8a4aeb7b95be9297` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.279943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] MFA Rules not processed for user `f2465e6ec8d94ee594e238d50fe29691`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:58.281510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.283636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9facbd9b3a69461e8a4aeb7b95be9297)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:58.285830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:58.302910 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2559/5170] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3281 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:58.312940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:58.362646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a2260-4e3c-40cf-b485-f991d20ca19c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:58.411090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2465e6ec8d94ee594e238d50fe29691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64e1050175b74aad9a721ed6f5e2efcb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1820674315', 'tempest-DelegatedRole-1812551333'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:58.411800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.412198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.412499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.413854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorizing `identity:create_trust()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:58.415072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:58.493352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9facbd9b3a69461e8a4aeb7b95be9297', 'request_id': 'req-6571c898-49ce-4545-8706-92274555e3d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.518721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9facbd9b3a69461e8a4aeb7b95be9297', 'request_id': 'req-6571c898-49ce-4545-8706-92274555e3d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.536106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9facbd9b3a69461e8a4aeb7b95be9297', 'request_id': 'req-6571c898-49ce-4545-8706-92274555e3d8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.568138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] The token cache is being invalidate because project 9facbd9b3a69461e8a4aeb7b95be9297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:58.568623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.571200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6571c898-49ce-4545-8706-92274555e3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.574528 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2612/5171] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:58 2022] DELETE /identity/v3/projects/9facbd9b3a69461e8a4aeb7b95be9297 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:10:58.585583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507707-6403-42ed-ab80-cab2c64d1793 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.585977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507707-6403-42ed-ab80-cab2c64d1793 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.586302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507707-6403-42ed-ab80-cab2c64d1793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.599309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2560/5172] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:10:58 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:58.600975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:58.607399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.651516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c308bc-6fd1-4a5f-96d8-61a221798887 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:58.684069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0c507707-6403-42ed-ab80-cab2c64d1793 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:58.685581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0c507707-6403-42ed-ab80-cab2c64d1793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.787403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2465e6ec8d94ee594e238d50fe29691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64e1050175b74aad9a721ed6f5e2efcb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1820674315', 'tempest-DelegatedRole-1812551333'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:58.788147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.788434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.788747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.789566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:58.794461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:58.811500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2561/5173] 10.222.0.52 () {58 vars in 1252 bytes} [Sat Sep 24 21:10:58 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=f2465e6ec8d94ee594e238d50fe29691 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:58.816975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:58.831334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:58.841762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341e2d-6316-4c57-8f3c-7679ea5334b5 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:58.854573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2465e6ec8d94ee594e238d50fe29691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64e1050175b74aad9a721ed6f5e2efcb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1820674315', 'tempest-DelegatedRole-1812551333'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:58.855261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.855531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.855947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] PATH_INFO: `/v3/OS-TRUST/trusts/c3da7bd22972430395b15c16769dde4b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.867759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2613/5174] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:58.869444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:58.869738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:58.870006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:58.894637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorizing `identity:delete_trust(trust_id=c3da7bd22972430395b15c16769dde4b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:58.898540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:58.916260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:10:58.920674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:58.960786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'c3da7bd22972430395b15c16769dde4b', 'request_id': 'req-a9761ef1-d653-46ed-a3b9-5f4362531dec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.961366 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2614/5175] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:10:58.961904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'c3da7bd22972430395b15c16769dde4b', 'request_id': 'req-a9761ef1-d653-46ed-a3b9-5f4362531dec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.964691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:58.988796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'c3da7bd22972430395b15c16769dde4b', 'request_id': 'req-a9761ef1-d653-46ed-a3b9-5f4362531dec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:58.997365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070d7d4-9d22-4ea4-aeaf-24ce11cf5106 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:58.999052 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2562/5176] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:10:58 2022] DELETE /identity/v3/OS-TRUST/trusts/c3da7bd22972430395b15c16769dde4b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:59.008735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.010441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.012321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.013142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.013526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.013896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.014763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:10:59.015213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.016943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.037596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9761ef1-d653-46ed-a3b9-5f4362531dec tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.060448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2615/5177] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:10:59.060448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.060790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2465e6ec8d94ee594e238d50fe29691', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '64e1050175b74aad9a721ed6f5e2efcb', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1820674315', 'tempest-DelegatedRole-1812551333'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.061456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.061586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.061795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] PATH_INFO: `/v3/OS-TRUST/trusts/c3da7bd22972430395b15c16769dde4b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.064231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Could not find trust: c3da7bd22972430395b15c16769dde4b.: keystone.exception.TrustNotFound: Could not find trust: c3da7bd22972430395b15c16769dde4b. Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 24 21:10:59.087444 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: c3da7bd22972430395b15c16769dde4b. Sep 24 21:10:59.088938 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:10:59.090854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2563/5178] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:10:59 2022] GET /identity/v3/OS-TRUST/trusts/c3da7bd22972430395b15c16769dde4b => generated 111 bytes in 83 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:10:59.093869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2afd9-a65a-4309-9b3c-eae4139a9d05 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.095759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.099227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.106771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.107665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.108021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.108368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.109024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.116850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.189504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2616/5179] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:10:59 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:10:59.191345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.286554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044ad5-852a-4ae4-8b5f-f5d76e000ee3 tempest-TrustsV3TestJSON-1193283477 tempest-user-1025355039] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.288008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:59.352433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149bb20-5ce0-4e50-9d2b-2bcfb812936b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.372424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.373035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.373316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.373594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.391693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.407589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.475123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.476450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.482937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.484047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/d0d27654f2564f229b1c4ed97bd79229` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.488753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d0d27654f2564f229b1c4ed97bd79229)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.497862 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2617/5180] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:10:59 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:10:59.498665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.498876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.582224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b00212-f0fa-4301-8eff-88a7471f37f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.597981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.599189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.599682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.600158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.601938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.603936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.659178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role d0d27654f2564f229b1c4ed97bd79229 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:59.659533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:59.665122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:59.667213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2564/5181] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:59 2022] DELETE /identity/v3/roles/d0d27654f2564f229b1c4ed97bd79229 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:59.672555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.674596 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2618/5182] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:59 2022] GET /identity/v3/roles => generated 1893 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:59.677945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.680036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.685157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.743964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3042722a-4b2a-4f56-8083-0d5d489253d9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.815853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:10:59.883747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecbdec-7181-4835-aba0-46dfbb7f5125 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.889326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.890297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.890676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.890970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/db08f56f1ca64a419e11ea69e68238b7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.892698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=db08f56f1ca64a419e11ea69e68238b7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.894434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.894600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:10:59.895227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:10:59.895510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:10:59.895785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:10:59.896426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:10:59.900918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:10:59.945751 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2619/5183] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:10:59 2022] GET /identity/v3/roles => generated 1655 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:10:59.945751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.956084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:10:59.984810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role db08f56f1ca64a419e11ea69e68238b7 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:10:59.985112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:59.987564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:10:59.991413 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2565/5184] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:10:59 2022] DELETE /identity/v3/roles/db08f56f1ca64a419e11ea69e68238b7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:10:59.992111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09aa7e-dcc4-4add-825c-aad33ea39f38 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:10:59.993908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:10:59.996131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.132986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:00.133641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.134056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.134201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] PATH_INFO: `/v3/projects/20425031dd4a4f40b8e6b1b75af25fff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.139432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20425031dd4a4f40b8e6b1b75af25fff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:00.141635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:00.182409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f328-9e70-40ba-b272-7061dd609b5d tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:00.189180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:00.189866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.190151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.190552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] PATH_INFO: `/v3/projects/67dd3aa3a7094a948ad9a8df00dd31a9/users/ab18fe421f554973afebd677c37b4c2a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.213353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67dd3aa3a7094a948ad9a8df00dd31a9, user_id=ab18fe421f554973afebd677c37b4c2a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:00.216364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:00.259314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2566/5185] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:10:59 2022] PUT /identity/v3/projects/67dd3aa3a7094a948ad9a8df00dd31a9/users/ab18fe421f554973afebd677c37b4c2a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:00.259314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:00.263043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20425031dd4a4f40b8e6b1b75af25fff', 'request_id': 'req-98d560b6-356e-45c5-988e-49b57c23e0e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.269132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.282175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20425031dd4a4f40b8e6b1b75af25fff', 'request_id': 'req-98d560b6-356e-45c5-988e-49b57c23e0e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.285771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:00.296052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20425031dd4a4f40b8e6b1b75af25fff', 'request_id': 'req-98d560b6-356e-45c5-988e-49b57c23e0e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.328941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] The token cache is being invalidate because project 20425031dd4a4f40b8e6b1b75af25fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:00.329277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.335920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98d560b6-356e-45c5-988e-49b57c23e0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.340148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2620/5186] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:10:59 2022] DELETE /identity/v3/projects/20425031dd4a4f40b8e6b1b75af25fff => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:00.345460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4b96-f8bc-4315-b2d1-fedaf03c9b52 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.345751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4b96-f8bc-4315-b2d1-fedaf03c9b52 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.346019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf4b96-f8bc-4315-b2d1-fedaf03c9b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.393204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0ccf4b96-f8bc-4315-b2d1-fedaf03c9b52 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:00.394563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0ccf4b96-f8bc-4315-b2d1-fedaf03c9b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.453823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc464c-bc7f-46dd-a751-1480c3fcbb83 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:00.477958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:00.478761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.479142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.479534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/f2465e6ec8d94ee594e238d50fe29691` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.480791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f2465e6ec8d94ee594e238d50fe29691)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:00.483316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:00.519448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:00.557080 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2621/5187] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:00.561705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.561705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.561705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.587266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2465e6ec8d94ee594e238d50fe29691'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.589170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2465e6ec8d94ee594e238d50fe29691'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.613315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2465e6ec8d94ee594e238d50fe29691'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.653790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2465e6ec8d94ee594e238d50fe29691'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.654982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] MFA Rules not processed for user `ab18fe421f554973afebd677c37b4c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:00.656744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.659057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb198466-ff75-4607-9963-59bd4b90ddf6 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2465e6ec8d94ee594e238d50fe29691'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:00.703218 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2567/5188] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:00 2022] DELETE /identity/v3/users/f2465e6ec8d94ee594e238d50fe29691 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:00.706658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:00.707121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:00.707408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:00.783884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:00.786327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.916051 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2622/5189] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:00.916980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:00.920624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:00.946028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360c09b-2577-4628-bb43-3045da18d644 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:00.983584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2568/5190] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:00.984877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:00.986478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:01.001574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d129f0-954c-48e0-944e-38a5fa9c7087 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:01.007639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.008287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.008570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.008848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.009589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.011698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.071146 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2569/5191] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:00 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:01.072276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.111297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.112236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.112536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.112830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/64e1050175b74aad9a721ed6f5e2efcb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.117702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=64e1050175b74aad9a721ed6f5e2efcb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.120230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.181333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:01.241688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e1050175b74aad9a721ed6f5e2efcb', 'request_id': 'req-af392bca-459a-44b9-b604-5771596e8654'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:01.253054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3803-7547-4b3f-8c4a-89250e2c4a3b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:01.256177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e1050175b74aad9a721ed6f5e2efcb', 'request_id': 'req-af392bca-459a-44b9-b604-5771596e8654'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:01.266401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e1050175b74aad9a721ed6f5e2efcb', 'request_id': 'req-af392bca-459a-44b9-b604-5771596e8654'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:01.295010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project 64e1050175b74aad9a721ed6f5e2efcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:01.295326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:01.296745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:01.298882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:01.299929 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2623/5192] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:00 2022] DELETE /identity/v3/projects/64e1050175b74aad9a721ed6f5e2efcb => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:01.301199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.308491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:01.408578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.409200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.409477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.409744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.410395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:01.410717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.412327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.442763 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2570/5193] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:01.443640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.456078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:01.501434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392bca-459a-44b9-b604-5771596e8654 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:01.526351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986305-de33-40af-9057-d965833f89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.534753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986305-de33-40af-9057-d965833f89bd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.535126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986305-de33-40af-9057-d965833f89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.535318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58986305-de33-40af-9057-d965833f89bd admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.540178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986305-de33-40af-9057-d965833f89bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.546860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58986305-de33-40af-9057-d965833f89bd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.621865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2624/5194] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:01 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:01.621865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986305-de33-40af-9057-d965833f89bd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.623891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58986305-de33-40af-9057-d965833f89bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:01.657766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58986305-de33-40af-9057-d965833f89bd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:01.671239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.671868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.672195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.672550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.673179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.675101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.679977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3c930-5549-44e1-82a4-0bc6010392fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:01.688817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:01.690150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:01.690719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:01.691268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:01.692221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:01.697609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:01.760279 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2625/5195] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:01 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:01.760279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.808692 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2571/5196] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:01 2022] POST /identity/v3/projects => generated 328 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:01.812609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:01.945148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dafb-6037-4875-89c4-056b90eff982 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.016052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:02.037242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdb98e-8003-431e-83cb-9606d49b63ce tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.053193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.053837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.054244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.055336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.055484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.057545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.096686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.097832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.098794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.100254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.104086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.110057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.112207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2572/5197] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:01 2022] POST /identity/v3/users => generated 287 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:02.116155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.139776 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2626/5198] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:02.143669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.170675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa6fe4-b0d0-489b-9da2-c082a5c8e15f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.185844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.188609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.191434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.193479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.196917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.202236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.222224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:02.232684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2627/5199] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:02.234152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.266292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2b0d9-31f5-4038-b3f3-aded89bceada admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.278904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.281680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.282992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.283504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] PATH_INFO: `/v3/projects/e105facdff49413391f9a3c0993fb0f0/users/c9a9afc1574f4ba3a61ad7ca46389866/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.294279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ebe9a-9c8d-4ffc-a77c-db885f813b82 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.301109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.302360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.302958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.303819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.304698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.307152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.340375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e105facdff49413391f9a3c0993fb0f0, user_id=c9a9afc1574f4ba3a61ad7ca46389866, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.346352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.374971 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2573/5200] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:02 2022] POST /identity/v3/users => generated 416 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:02.375969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.386165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2628/5201] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:02 2022] PUT /identity/v3/projects/e105facdff49413391f9a3c0993fb0f0/users/c9a9afc1574f4ba3a61ad7ca46389866/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:02.387032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.408325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b4624-49a9-4d6c-b257-3fa48aa2bc3c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.529612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98848d1e-f97d-439f-8316-849cc7ee966a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.538293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.539089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.539203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.539342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.539509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.539509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.539685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.539901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.540107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.540525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.541784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.542140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.569157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2629/5202] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:02 2022] POST /identity/v3/roles => generated 244 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:02.573995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.586488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2574/5203] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:02.587761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.596172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee4292-948c-4a00-9c7f-109e5be978b1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.602388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.603042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.603476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.603914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.604651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.606757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.624598 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2630/5204] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:02 2022] GET /identity/v3/roles => generated 1655 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:02.624598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.638454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae051889-7419-4414-b87c-00ba1c158e66 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.649367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.650328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.650917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.651327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.652009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.653572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b0bea-4a95-4104-ba92-823fa5db1c15 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.653787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.680232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.680849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.681136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.681588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] PATH_INFO: `/v3/projects/95033b8b74ad4b34921560d15bd2f6c6/users/59239200db78440c94486bca4ea39831/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.693344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2575/5205] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:02 2022] POST /identity/v3/roles => generated 246 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:02.693344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.723387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f744f6-4f84-4ea9-a49d-bab8a3ce8c6e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:02.733566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:02.735828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.736983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.737472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] PATH_INFO: `/v3/projects/e105facdff49413391f9a3c0993fb0f0/users/c9a9afc1574f4ba3a61ad7ca46389866/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.743010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e105facdff49413391f9a3c0993fb0f0, user_id=c9a9afc1574f4ba3a61ad7ca46389866, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.746973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.763273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95033b8b74ad4b34921560d15bd2f6c6, user_id=59239200db78440c94486bca4ea39831, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:02.766052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e330104-e365-4303-806a-7fb461a2fc74 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:02.788990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2576/5206] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:02 2022] PUT /identity/v3/projects/e105facdff49413391f9a3c0993fb0f0/users/c9a9afc1574f4ba3a61ad7ca46389866/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:02.790275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:02.808875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2631/5207] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:02 2022] PUT /identity/v3/projects/95033b8b74ad4b34921560d15bd2f6c6/users/59239200db78440c94486bca4ea39831/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:02.810761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9afe9-ef9b-4437-8e0f-678da2503033 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:02.810761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9afe9-ef9b-4437-8e0f-678da2503033 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:02.810761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9afe9-ef9b-4437-8e0f-678da2503033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:02.888212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-39f9afe9-ef9b-4437-8e0f-678da2503033 None None] MFA Rules not processed for user `c9a9afc1574f4ba3a61ad7ca46389866`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:02.890301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-39f9afe9-ef9b-4437-8e0f-678da2503033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:02.970375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:03.037175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a07d52-5077-4208-adba-6fd5674bf453 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:03.071163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2632/5208] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:03.073264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.073817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.074314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.135708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] MFA Rules not processed for user `59239200db78440c94486bca4ea39831`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:03.137610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:03.155885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:03.229163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:03.229826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.230109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.232548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/168cf7e9afd748ca9944d5680a044982/users/685e0efed33a4c91956958afa533d3ff/roles/28da31450d134cf18c3fe5c443c9b81b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.286311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=168cf7e9afd748ca9944d5680a044982, user_id=685e0efed33a4c91956958afa533d3ff, role_id=28da31450d134cf18c3fe5c443c9b81b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:03.294709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:03.310318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2633/5209] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:03.311370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:03.339906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43457c-976c-460d-909b-bea9fa2d2b87 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:03.350822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2577/5210] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:02 2022] PUT /identity/v3/projects/168cf7e9afd748ca9944d5680a044982/users/685e0efed33a4c91956958afa533d3ff/roles/28da31450d134cf18c3fe5c443c9b81b => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:03.351795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:03.476855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9a9afc1574f4ba3a61ad7ca46389866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e105facdff49413391f9a3c0993fb0f0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:03.477732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.478091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.478439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.479276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:03.479582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:03.487335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:03.512200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5f22-1ab1-4672-9183-a08d25ae5e37 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:03.513689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2634/5211] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:03.514783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:03.531083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:03.532676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.532676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.532676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.532676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:03.541601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:03.614855 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2578/5212] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:03 2022] POST /identity/v3/projects => generated 373 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:03.615780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:03.656990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:03.729512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a78c7d-f8cd-42d8-800b-7abbd27be853 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:03.879600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf18b1-4687-40aa-a9aa-19cc0a2f841b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:03.901133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9a9afc1574f4ba3a61ad7ca46389866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e105facdff49413391f9a3c0993fb0f0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:03.901871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.902045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.902462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.903165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:03.905120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:03.907301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:03.908106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:03.908492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:03.908868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/168cf7e9afd748ca9944d5680a044982/users/685e0efed33a4c91956958afa533d3ff/roles/aee589f2dba74badaae0ddca44ccb016` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:03.916723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=168cf7e9afd748ca9944d5680a044982, user_id=685e0efed33a4c91956958afa533d3ff, role_id=aee589f2dba74badaae0ddca44ccb016)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:03.919870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:03.953953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2635/5213] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:03 2022] PUT /identity/v3/projects/168cf7e9afd748ca9944d5680a044982/users/685e0efed33a4c91956958afa533d3ff/roles/aee589f2dba74badaae0ddca44ccb016 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:03.955130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.007459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2579/5214] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:03 2022] POST /identity/v3/projects => generated 350 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:04.009446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.076315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:04.109016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437490b-bc82-4a80-950b-896e93fe8bf5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.131636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd01cbb-aba2-441a-87d2-efac0839181e tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.145291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.146184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.149000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.149401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.151990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.155100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.214260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2580/5215] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:04 2022] GET /identity/v3/users => generated 8619 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:04.216615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.265862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.266698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.267103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.267446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.268459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.270260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8406-508f-41af-9bd9-82a9e860b14c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.327049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7bcf-f909-40c9-a74d-7c51f5cb3f87 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.348741 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2636/5216] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:03 2022] POST /identity/v3/users => generated 293 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:04.348987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9a9afc1574f4ba3a61ad7ca46389866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e105facdff49413391f9a3c0993fb0f0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.349123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.349228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.349228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.349228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.349228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:04.350216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.350613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.350743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.352282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.392731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2581/5217] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:04.392731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.406530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42453746-ca0e-4b26-aefd-757a96572744 tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.415879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.416670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.417003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.417284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.417929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.419961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.447973 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2582/5218] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:04 2022] GET /identity/v3/roles => generated 1892 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:04.451738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.492135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58616dd-aac7-4906-9c54-f583f2b1b0b0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.510000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.510745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] MFA Rules not processed for user `685e0efed33a4c91956958afa533d3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:04.511680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.513563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.513854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] PATH_INFO: `/v3/projects/05814d65d2844f6b9077b95dabbb5bf3/users/9f981e6734744a37a42e6fb761fd6c79/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.515442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:04.541833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:04.558198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05814d65d2844f6b9077b95dabbb5bf3, user_id=9f981e6734744a37a42e6fb761fd6c79, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.564764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.594600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2583/5219] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:04 2022] PUT /identity/v3/projects/05814d65d2844f6b9077b95dabbb5bf3/users/9f981e6734744a37a42e6fb761fd6c79/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:04.595794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.602327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:04.756826 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2637/5220] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:04.757662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:04.774372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205da5a0-f5bc-4783-b570-2e8172c22d24 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.790081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.790878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.791260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.791599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.809855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456061-0d5c-48a4-9724-ad7c31f4c4e8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:04.892394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:04.938402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.940259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8963650-3083-4428-adc5-61e7a00b13bc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.941150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:04.942017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.942584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.943090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.945267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:04.948490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:04.967579 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2584/5221] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:04.969077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:04.969352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:04.969711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:04.982874 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2638/5222] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:04 2022] GET /identity/v3/roles => generated 1892 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:04.984361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.010232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62968c-0ed8-44cd-bd91-5f4056e489f3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.019550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] MFA Rules not processed for user `685e0efed33a4c91956958afa533d3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:05.020773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:05.022733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.024764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.025443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.025887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] PATH_INFO: `/v3/projects/05814d65d2844f6b9077b95dabbb5bf3/users/9f981e6734744a37a42e6fb761fd6c79/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.035110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05814d65d2844f6b9077b95dabbb5bf3, user_id=9f981e6734744a37a42e6fb761fd6c79, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.048080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01712867-5d14-4805-8b1a-f4a474874ec5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.109116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2639/5223] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:04 2022] PUT /identity/v3/projects/05814d65d2844f6b9077b95dabbb5bf3/users/9f981e6734744a37a42e6fb761fd6c79/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:05.119088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.119859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.120161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.195627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2585/5224] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:05.199697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.202773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] MFA Rules not processed for user `9f981e6734744a37a42e6fb761fd6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:05.204050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:05.215385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd71e22-feb4-4b52-807d-816fbcc07a87 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.317319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.322687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.322981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.323262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.324030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:create_trust()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.325759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.424239 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2640/5225] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:11:05.425594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.448294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af8f74-4cc4-47d7-8922-cdd3998dd85f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.464978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.466594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.467146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.467645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.468533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.471574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.555620 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2641/5226] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/services => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:05.557198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.568892 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2586/5227] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/OS-TRUST/trusts => generated 859 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:05.574128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.595663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055eb69-c40d-4cdd-a4e7-56dc0e0c8f57 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.605174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:05.629111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.629846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.630271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.630583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.651194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:get_trust(trust_id=91ce9c6815bb43acb978e128f5eac635)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.652759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.654739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:05.664429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ec88e-d330-4279-a80c-57c31edf3b15 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.671080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2587/5228] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:05 2022] GET /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635 => generated 859 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:05.675877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.689902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.693062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.696859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.699192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.703378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.706920 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:05.706920 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:05.706920 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:05.708605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.728114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae52f3-3f77-4422-b516-db9d8ff43aec tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.749541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.750258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.750555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.750819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.759915 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2642/5229] 10.222.0.52 () {60 vars in 1213 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:05.764027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.788568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=91ce9c6815bb43acb978e128f5eac635)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.790457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.812603 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2588/5230] 10.222.0.52 () {58 vars in 1269 bytes} [Sat Sep 24 21:11:05 2022] GET /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles => generated 371 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:11:05.812807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901bb49-2b2e-4997-b365-5de20fd61c09 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.814196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.832979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.833476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14504c-f146-47f2-a900-1123d7c0fae4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.837010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.838486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.838790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.841220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.848153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.855385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.856157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.856544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.856871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/28da31450d134cf18c3fe5c443c9b81b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.876132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:get_role_for_trust(trust_id=91ce9c6815bb43acb978e128f5eac635, role_id=28da31450d134cf18c3fe5c443c9b81b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.878154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.885363 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2643/5231] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/services => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:05.888211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.902687 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2589/5232] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:05 2022] GET /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/28da31450d134cf18c3fe5c443c9b81b => generated 244 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:05.912201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.929497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de2ee-e2b0-4914-b983-9ef86005f689 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.943012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cae9c-9493-4516-b3af-834b85efb455 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:05.956796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.957419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.957713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.958092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:05.958974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:05.961908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:05.988646 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2644/5233] 10.222.0.52 () {60 vars in 1213 bytes} [Sat Sep 24 21:11:05 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:05.992909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:05.995313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:05.995703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:05.996034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:05.996248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/28da31450d134cf18c3fe5c443c9b81b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.007988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177d593-4769-4b29-b2ee-f485286625ae tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.021386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.022806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.023370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.023614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.024231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.027211 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:06.027211 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:06.027211 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:06.030139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.033775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:get_role_for_trust(trust_id=91ce9c6815bb43acb978e128f5eac635, role_id=28da31450d134cf18c3fe5c443c9b81b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.035652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.060403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2645/5234] 10.222.0.52 () {58 vars in 1191 bytes} [Sat Sep 24 21:11:05 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 764 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:11:06.066948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.072964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2590/5235] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:05 2022] HEAD /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/28da31450d134cf18c3fe5c443c9b81b => generated 0 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 24 21:11:06.077409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.103800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018713d-88c6-4cb8-a894-6cd1bd0bb708 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.104826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f68b-5695-4e0a-84b5-ffd89e208633 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.117714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.118559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.118853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.119171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/aee589f2dba74badaae0ddca44ccb016` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.127540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.129771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.130883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.132862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.134189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=29f16cf279d5491dabea3539b5596c8b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.138111 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:06.138111 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:06.138111 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:06.139206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.149116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:get_role_for_trust(trust_id=91ce9c6815bb43acb978e128f5eac635, role_id=aee589f2dba74badaae0ddca44ccb016)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.157312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Could not find role: aee589f2dba74badaae0ddca44ccb016.: keystone.exception.RoleNotFound: Could not find role: aee589f2dba74badaae0ddca44ccb016. Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:06.180055 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: aee589f2dba74badaae0ddca44ccb016. Sep 24 21:11:06.181338 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:06.186008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2646/5236] 10.222.0.52 () {58 vars in 1290 bytes} [Sat Sep 24 21:11:06 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b => generated 333 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:06.187066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.187693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2591/5237] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:06 2022] GET /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/aee589f2dba74badaae0ddca44ccb016 => generated 110 bytes in 111 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:06.188737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.220180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968079-42ce-46f2-b2e8-1d1bf92b23a5 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.230146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.231023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.231433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.231792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/aee589f2dba74badaae0ddca44ccb016` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.234095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1cacd-f765-4795-988c-ff635241b21d tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.245619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.246396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.246714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.250361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.251476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:get_role_for_trust(trust_id=91ce9c6815bb43acb978e128f5eac635, role_id=aee589f2dba74badaae0ddca44ccb016)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.252548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=29f16cf279d5491dabea3539b5596c8b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.253599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.259469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Could not find role: aee589f2dba74badaae0ddca44ccb016.: keystone.exception.RoleNotFound: Could not find role: aee589f2dba74badaae0ddca44ccb016. Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:06.278382 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: aee589f2dba74badaae0ddca44ccb016. Sep 24 21:11:06.280049 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:06.292477 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2592/5238] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:06 2022] HEAD /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635/roles/aee589f2dba74badaae0ddca44ccb016 => generated 0 bytes in 101 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 24 21:11:06.293524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.301083 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2647/5239] 10.222.0.52 () {58 vars in 1291 bytes} [Sat Sep 24 21:11:06 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b => generated 0 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 24 21:11:06.304370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.318396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc34121-4734-4090-9888-c8e11c1066c4 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.323979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.324759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.325082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.325344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.344465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorizing `identity:delete_trust(trust_id=91ce9c6815bb43acb978e128f5eac635)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.345057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200f53-befd-43a4-9dca-9d1d64f31cad tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.346133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.363837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.364649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.364952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.365224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.365938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=29f16cf279d5491dabea3539b5596c8b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.368972 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:06.368972 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:06.368972 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:06.371358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.384325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '91ce9c6815bb43acb978e128f5eac635', 'request_id': 'req-592b669e-c4c6-4c60-ae13-be4718dc3317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:06.386077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '91ce9c6815bb43acb978e128f5eac635', 'request_id': 'req-592b669e-c4c6-4c60-ae13-be4718dc3317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:06.400473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '91ce9c6815bb43acb978e128f5eac635', 'request_id': 'req-592b669e-c4c6-4c60-ae13-be4718dc3317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:06.412146 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2648/5240] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:06.416560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.417532 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2593/5241] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:06.420812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:06.429991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.435274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:06.450169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b669e-c4c6-4c60-ae13-be4718dc3317 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.460209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '685e0efed33a4c91956958afa533d3ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168cf7e9afd748ca9944d5680a044982', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2136165132', 'tempest-NotDelegatedRole-468879866'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.461884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.462174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.464380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] PATH_INFO: `/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.471026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a556a36-8344-4e31-ac9d-6d1d393db3b1 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.484673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.485306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.485595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.485874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.493315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Could not find trust: 91ce9c6815bb43acb978e128f5eac635.: keystone.exception.TrustNotFound: Could not find trust: 91ce9c6815bb43acb978e128f5eac635. Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 24 21:11:06.503710 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 91ce9c6815bb43acb978e128f5eac635. Sep 24 21:11:06.504708 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:06.506995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.507667 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2594/5242] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:06 2022] GET /identity/v3/OS-TRUST/trusts/91ce9c6815bb43acb978e128f5eac635 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:06.512465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.517310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:06.528554 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2649/5243] 10.222.0.52 () {58 vars in 1191 bytes} [Sat Sep 24 21:11:06 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:06.529002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.561787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7febaa6-5203-45d4-9fd4-0b8583978fb5 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.571313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.572296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.572446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.572750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.573343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=29f16cf279d5491dabea3539b5596c8b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.575769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Could not find Endpoint Group: 29f16cf279d5491dabea3539b5596c8b.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 29f16cf279d5491dabea3539b5596c8b. Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete Sep 24 21:11:06.592402 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: 29f16cf279d5491dabea3539b5596c8b. Sep 24 21:11:06.593524 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:06.601252 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2650/5244] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/29f16cf279d5491dabea3539b5596c8b => generated 120 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:06.616489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.660692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66384-df6f-4334-afe4-d80554f54ace tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.667743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.668384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.668726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.668996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services/66fe3da70add4101927d0ffcf6fa7923` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.669639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_service(service_id=66fe3da70add4101927d0ffcf6fa7923)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.671635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.731879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099e56-6952-49e0-8a88-786c0884cf31 tempest-TrustsV3TestJSON-763601280 tempest-user-1353336348] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.733892 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2651/5245] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/services/66fe3da70add4101927d0ffcf6fa7923 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:06.739694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.765176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:06.801326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b9be-a79f-4d56-aaf9-bb3ef2c36465 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.808033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.811301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.811992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.812279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.812559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.813322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.814302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.815040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.815163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.815571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/aee589f2dba74badaae0ddca44ccb016` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.825875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=aee589f2dba74badaae0ddca44ccb016)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.830254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.849653 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2652/5246] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:06 2022] POST /identity/v3/services => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:06.853375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.921863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1a9f3-842c-4a3c-99cf-987520f34522 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:06.936267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role aee589f2dba74badaae0ddca44ccb016 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:06.938777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:06.939440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:06.940488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:06.940774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:06.941112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:06.941747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:06.945497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:06.949496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2595/5247] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/roles/aee589f2dba74badaae0ddca44ccb016 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:06.949660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:06.952242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.954089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:06.970905 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2653/5248] 10.222.0.52 () {60 vars in 1213 bytes} [Sat Sep 24 21:11:06 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:06.976659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:06.979464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.013126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea6b25-21bd-4d59-87d2-01071c7b67ca tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.117013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc5448-2050-4053-9690-bcb5834d38fe tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.140083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:07.142355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.143139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.143520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.144647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/28da31450d134cf18c3fe5c443c9b81b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.147599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=28da31450d134cf18c3fe5c443c9b81b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.154456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.155370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.156236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.156549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.156841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.157705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.160908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.189593 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2654/5249] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:06 2022] POST /identity/v3/services => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:07.190633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.192335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.227593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role 28da31450d134cf18c3fe5c443c9b81b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:07.227963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.230410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.234619 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2596/5250] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:06 2022] DELETE /identity/v3/roles/28da31450d134cf18c3fe5c443c9b81b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:07.235215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.242655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.352614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b0f69-9256-4fd3-834b-d30c915c7c9f tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.358688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fd67-ad4a-4133-9f04-508ad559a1df tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.376321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.377452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.378040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.378531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/faf6eb756cda4b949e148e59109f17a4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.379401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=faf6eb756cda4b949e148e59109f17a4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.380871 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:07.380871 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:07.380871 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:07.382823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.402322 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2597/5251] 10.222.0.52 () {60 vars in 1313 bytes} [Sat Sep 24 21:11:07 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/faf6eb756cda4b949e148e59109f17a4 => generated 335 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:07.403690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.408388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.419283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:07.436139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.437043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.437333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.437609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.441211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.517761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799a561-03b3-4670-aa0a-7aba5f06ce27 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.523432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.524122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.524389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.524723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/685e0efed33a4c91956958afa533d3ff` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.525901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=685e0efed33a4c91956958afa533d3ff)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.528078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.558639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.562182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.603586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685e0efed33a4c91956958afa533d3ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.605456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685e0efed33a4c91956958afa533d3ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.619411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685e0efed33a4c91956958afa533d3ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.629599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685e0efed33a4c91956958afa533d3ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.633696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685e0efed33a4c91956958afa533d3ff'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:07.640047 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2655/5252] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3639 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:07.643016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.683301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bd2fb-e37e-41ef-bcca-488c9b3691da service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.703164 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2598/5253] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:07 2022] DELETE /identity/v3/users/685e0efed33a4c91956958afa533d3ff => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:07.708400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.708737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.715481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.718974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.719634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services/60f9765fa9c946ef919c67d323c67b73` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.721129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_service(service_id=60f9765fa9c946ef919c67d323c67b73)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.723491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.724088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.784329 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2656/5254] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:07 2022] DELETE /identity/v3/services/60f9765fa9c946ef919c67d323c67b73 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:07.789728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:07.793235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:07.840255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5854a-904e-4454-8d0a-269718612abe tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.853133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:07.853634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fb756-3299-49bf-8390-41cac91712d4 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:07.896509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.897422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.900422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.900731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/168cf7e9afd748ca9944d5680a044982` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.904422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=168cf7e9afd748ca9944d5680a044982)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.910468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.920873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:07.969102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:07.969791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:07.970115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:07.970418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/faf6eb756cda4b949e148e59109f17a4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:07.971531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=faf6eb756cda4b949e148e59109f17a4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:07.974131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:07.990103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2657/5255] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:11:07 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/faf6eb756cda4b949e148e59109f17a4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:07.994928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.017805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98002e-fe7f-4626-ac7a-67de7e6f3fea tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.021228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168cf7e9afd748ca9944d5680a044982', 'request_id': 'req-af3be711-ffce-4c16-a57c-dea306c3aa88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.043952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168cf7e9afd748ca9944d5680a044982', 'request_id': 'req-af3be711-ffce-4c16-a57c-dea306c3aa88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.059367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168cf7e9afd748ca9944d5680a044982', 'request_id': 'req-af3be711-ffce-4c16-a57c-dea306c3aa88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.085221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project 168cf7e9afd748ca9944d5680a044982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:08.086850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.088382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.092139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2599/5256] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:07 2022] DELETE /identity/v3/projects/168cf7e9afd748ca9944d5680a044982 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:08.099268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.110569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:08.202017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.203202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.203539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.203866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services/af9a4aa9193e49479fd4f059f147835c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.207411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_service(service_id=af9a4aa9193e49479fd4f059f147835c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.210389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.260329 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2658/5257] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:07 2022] DELETE /identity/v3/services/af9a4aa9193e49479fd4f059f147835c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:08.264987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.266760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:08.277064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3be711-ffce-4c16-a57c-dea306c3aa88 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.302811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.306544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.307372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.307891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.308740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.312771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.331819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1e52-234d-4c3c-a1d5-c8b8278146f6 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.337030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.337683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.337968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.338247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/443a14e6629e4e2ba2b541d0de230334` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.339082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=443a14e6629e4e2ba2b541d0de230334)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.340460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.367689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2600/5258] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:08 2022] POST /identity/v3/projects => generated 328 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:08.371982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.372134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2659/5259] 10.222.0.52 () {58 vars in 1293 bytes} [Sat Sep 24 21:11:08 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/443a14e6629e4e2ba2b541d0de230334 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:08.376295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.396990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b8bbf-9147-4e12-bf93-0123ec789bf3 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.504504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfccad-13f3-46e7-96aa-74ea3b9a948f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.511961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.512640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.512860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.513875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.514686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.516641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.519311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f981e6734744a37a42e6fb761fd6c79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '05814d65d2844f6b9077b95dabbb5bf3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.520016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.520265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.520562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] PATH_INFO: `/v3/services/6483b65f98fc46acabf3a842b93a2582` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.521147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorizing `identity:delete_service(service_id=6483b65f98fc46acabf3a842b93a2582)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.522879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.525181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:08.563820 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2660/5260] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:08 2022] DELETE /identity/v3/services/6483b65f98fc46acabf3a842b93a2582 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:08.564522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:08.567593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.573829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:08.596522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2601/5261] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:08 2022] POST /identity/v3/users => generated 413 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:08.601031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.632692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d714-561f-4150-9886-6689f6f9e84c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.685447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.686543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.686845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97973fc4-f218-4e4a-8d7d-bb15412f2a39 tempest-EndPointGroupsTest-1508998082 tempest-EndPointGroupsTest-1508998082-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.687316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.687822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.688581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.690305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.693022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.694606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.695539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.696005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] PATH_INFO: `/v3/users/59239200db78440c94486bca4ea39831` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.698050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59239200db78440c94486bca4ea39831)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.701131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.723220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2602/5262] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:08 2022] POST /identity/v3/roles => generated 244 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:08.724391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.751236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3417-6afe-4656-98b2-d98b7d9cc71a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.757832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:08.758938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.759292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.765091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.768722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:08.775476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:08.789081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59239200db78440c94486bca4ea39831'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.790028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59239200db78440c94486bca4ea39831'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.808957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59239200db78440c94486bca4ea39831'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.813653 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2603/5263] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:08 2022] POST /identity/v3/roles => generated 247 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:08.817966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:08.821149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:08.830714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59239200db78440c94486bca4ea39831'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.831560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6edc0d39-cf4b-4b14-8255-c0e4e3196d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59239200db78440c94486bca4ea39831'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:08.868234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2661/5264] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:08 2022] DELETE /identity/v3/users/59239200db78440c94486bca4ea39831 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:08.881972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:08.881972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:08.881972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:08.890415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f9c38-85b9-4b3f-98f9-cdc15bd23499 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:08.957566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:08.963790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:09.122871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:09.128529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:09.129365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.130265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.130688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/1cf7180995f64a52adccecb3033eff59/users/be4f982c5ddf4073a7c537809ff9812e/roles/b7048d132ca1430683c655b40553492e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:09.174717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1cf7180995f64a52adccecb3033eff59, user_id=be4f982c5ddf4073a7c537809ff9812e, role_id=b7048d132ca1430683c655b40553492e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:09.176844 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2662/5265] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:09.177520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:09.181289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:09.194841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:09.212836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2604/5266] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:08 2022] PUT /identity/v3/projects/1cf7180995f64a52adccecb3033eff59/users/be4f982c5ddf4073a7c537809ff9812e/roles/b7048d132ca1430683c655b40553492e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:09.219081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:09.410884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc240-3729-4310-b6cf-bbd60dd1da6c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:09.418919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5cb97-f944-4323-ba3f-4b46941d0217 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:09.427574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:09.429168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.429650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.430055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/1cf7180995f64a52adccecb3033eff59/users/be4f982c5ddf4073a7c537809ff9812e/roles/d14be1e2c0634e6281960a4a454e61aa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:09.432837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1cf7180995f64a52adccecb3033eff59, user_id=be4f982c5ddf4073a7c537809ff9812e, role_id=d14be1e2c0634e6281960a4a454e61aa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:09.435759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:09.502769 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2605/5267] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:09 2022] PUT /identity/v3/projects/1cf7180995f64a52adccecb3033eff59/users/be4f982c5ddf4073a7c537809ff9812e/roles/d14be1e2c0634e6281960a4a454e61aa => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:09.503687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:09.510102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:09.560572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:09.561825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.562096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.562375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:09.672017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:09.698069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:09.699825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:09.736948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2663/5268] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:09.737916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:09.746003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da765d0-a1f0-45b5-86d6-6da3842c39f8 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:09.754367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:09.755168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.756892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.756892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:09.759104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:09.771048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79164a-ff35-4c87-a898-f964d07ddf6f service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:09.777096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:09.782549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b1df0-2639-4fed-8bb9-59f1f3132337 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:09.819285 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2606/5269] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:09.871359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:09.891038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:09.897768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.897768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.897768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:09.897768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:09.914655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe779bfe-d2ec-4a7e-822c-b889a0ae4b43 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:09.949056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2664/5270] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:09 2022] GET /identity/v3/users => generated 8622 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:09.954691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:09.955281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:09.955281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.050325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] MFA Rules not processed for user `be4f982c5ddf4073a7c537809ff9812e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:10.059140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:10.185247 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2607/5271] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:10.185897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:10.186867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:10.187586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.241530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] MFA Rules not processed for user `be4f982c5ddf4073a7c537809ff9812e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:10.243195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:10.253560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2665/5272] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:10.259211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:10.271491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841af13-af6b-4364-8aab-1b81ce77ae1f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:10.288532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be4f982c5ddf4073a7c537809ff9812e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cf7180995f64a52adccecb3033eff59', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1553639139', 'tempest-NotDelegatedRole-1550474051'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:10.289784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:10.290119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:10.290408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.291671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] RBAC: Authorizing `identity:create_trust()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:10.296214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 44, in parse_isotime Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) Sep 24 21:11:10.456362 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 46, in parse_isotime Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise ValueError(str(e)) Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post Sep 24 21:11:10.457423 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. Sep 24 21:11:10.458759 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:10.461654 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2608/5273] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:11:10 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 203 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:11:10.465493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:10.484644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe0c4c-442e-4a75-85bc-ba4e7ca2eb81 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:10.495378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:10.496048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:10.496312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:10.496586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/d14be1e2c0634e6281960a4a454e61aa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.499647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d14be1e2c0634e6281960a4a454e61aa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:10.501275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:10.561380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role d14be1e2c0634e6281960a4a454e61aa has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:10.561998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:10.562905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:10.572130 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2666/5274] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:10 2022] DELETE /identity/v3/roles/d14be1e2c0634e6281960a4a454e61aa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:10.572270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:10.574076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:10.595465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1ada1-1407-4dd7-b8f4-165e88bc1cca tempest-TrustsV3TestJSON-924994546 tempest-user-666511639] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:10.662795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:10.694614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:10.695646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:10.696163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:10.696451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/b7048d132ca1430683c655b40553492e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.698022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=b7048d132ca1430683c655b40553492e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:10.700227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:10.804744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role b7048d132ca1430683c655b40553492e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:10.805163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:10.806070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:10.810221 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2609/5275] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:10 2022] DELETE /identity/v3/roles/b7048d132ca1430683c655b40553492e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:10.814666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:10.834133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3589196-e6c9-4ec7-af06-b2c2cb696b1e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:10.932611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:10.936944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:10.987385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:10.988102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:10.988400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:10.988682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/be4f982c5ddf4073a7c537809ff9812e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:10.992329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=be4f982c5ddf4073a7c537809ff9812e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:10.996715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.088022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4f982c5ddf4073a7c537809ff9812e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.088867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4f982c5ddf4073a7c537809ff9812e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.111114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8b3a-3f45-433d-8b89-fab03985599f tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:11.118296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9a9afc1574f4ba3a61ad7ca46389866', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e105facdff49413391f9a3c0993fb0f0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:11.120119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:11.120531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:11.121139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] PATH_INFO: `/v3/projects/a69a10e14b574268afe49d54fd838d84` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:11.122999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a69a10e14b574268afe49d54fd838d84)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:11.124139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4f982c5ddf4073a7c537809ff9812e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.126222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.167557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4f982c5ddf4073a7c537809ff9812e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.168586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4f982c5ddf4073a7c537809ff9812e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.198825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69a10e14b574268afe49d54fd838d84', 'request_id': 'req-033930de-b443-4aa6-b924-950f30bdc78f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.211120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69a10e14b574268afe49d54fd838d84', 'request_id': 'req-033930de-b443-4aa6-b924-950f30bdc78f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.233581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69a10e14b574268afe49d54fd838d84', 'request_id': 'req-033930de-b443-4aa6-b924-950f30bdc78f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.239598 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2667/5276] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:10 2022] DELETE /identity/v3/users/be4f982c5ddf4073a7c537809ff9812e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:11.241794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:11.246519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:11.274203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] The token cache is being invalidate because project a69a10e14b574268afe49d54fd838d84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:11.274582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.275854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.286615 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2610/5277] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:10 2022] DELETE /identity/v3/projects/a69a10e14b574268afe49d54fd838d84 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:11.288356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:11.295355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:11.317901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:11.392423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:11.433616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192a444-a017-4d24-818a-05bda81dc342 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:11.445148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:11.445910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:11.446253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:11.446681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] PATH_INFO: `/v3/projects/95033b8b74ad4b34921560d15bd2f6c6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:11.455623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95033b8b74ad4b34921560d15bd2f6c6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:11.457823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.461438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033930de-b443-4aa6-b924-950f30bdc78f tempest-QuotasNegativeTest-1372473710 tempest-QuotasNegativeTest-1372473710-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:11.468160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:11.469089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:11.469639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:11.470594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] PATH_INFO: `/v3/users/370b057b12c148cdbb673ba6c06bd6b4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:11.472052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370b057b12c148cdbb673ba6c06bd6b4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:11.475801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.539871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95033b8b74ad4b34921560d15bd2f6c6', 'request_id': 'req-4e0cdf54-481d-4168-9db2-ba7d30efb445'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.547819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95033b8b74ad4b34921560d15bd2f6c6', 'request_id': 'req-4e0cdf54-481d-4168-9db2-ba7d30efb445'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.550348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370b057b12c148cdbb673ba6c06bd6b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.551879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370b057b12c148cdbb673ba6c06bd6b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.558308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95033b8b74ad4b34921560d15bd2f6c6', 'request_id': 'req-4e0cdf54-481d-4168-9db2-ba7d30efb445'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.563164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370b057b12c148cdbb673ba6c06bd6b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.572151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370b057b12c148cdbb673ba6c06bd6b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.573025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370b057b12c148cdbb673ba6c06bd6b4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.582766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] The token cache is being invalidate because project 95033b8b74ad4b34921560d15bd2f6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:11.583121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.584592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.587387 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2668/5278] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:11 2022] DELETE /identity/v3/projects/95033b8b74ad4b34921560d15bd2f6c6 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:11.588197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:11.589523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:11.601551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2611/5279] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:11 2022] DELETE /identity/v3/users/370b057b12c148cdbb673ba6c06bd6b4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:11.602774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:11.604593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:11.709062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cdf54-481d-4168-9db2-ba7d30efb445 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:11.743937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bee3e-74b4-4e3d-81b9-159cc2908c35 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:11.756419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:11.757459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:11.757743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:11.758017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] PATH_INFO: `/v3/users/ab18fe421f554973afebd677c37b4c2a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:11.762038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab18fe421f554973afebd677c37b4c2a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:11.764041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.838142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:11.862257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:11.863151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:11.863533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:11.863962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/1cf7180995f64a52adccecb3033eff59` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:11.866349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1cf7180995f64a52adccecb3033eff59)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:11.869333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:11.887294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab18fe421f554973afebd677c37b4c2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.888176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab18fe421f554973afebd677c37b4c2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.896357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:11.908763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab18fe421f554973afebd677c37b4c2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.923455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab18fe421f554973afebd677c37b4c2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.924367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab18fe421f554973afebd677c37b4c2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:11.980897 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2612/5280] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:11 2022] DELETE /identity/v3/users/ab18fe421f554973afebd677c37b4c2a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:11.984832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:11.990939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.015030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf7180995f64a52adccecb3033eff59', 'request_id': 'req-82bb6c5e-35b7-467f-b954-f7207d5b57dd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.027567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf7180995f64a52adccecb3033eff59', 'request_id': 'req-82bb6c5e-35b7-467f-b954-f7207d5b57dd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.039879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf7180995f64a52adccecb3033eff59', 'request_id': 'req-82bb6c5e-35b7-467f-b954-f7207d5b57dd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.062902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project 1cf7180995f64a52adccecb3033eff59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:12.065016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.067488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.070782 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2669/5281] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:11 2022] DELETE /identity/v3/projects/1cf7180995f64a52adccecb3033eff59 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:12.075540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:12.080897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.145242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdef779-46d0-4199-9c0d-f551a233fc57 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:12.162067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:12.164890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.166717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.167011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] PATH_INFO: `/v3/users/9f981e6734744a37a42e6fb761fd6c79` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.171913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f981e6734744a37a42e6fb761fd6c79)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:12.181543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:12.281742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6c5e-35b7-467f-b954-f7207d5b57dd tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:12.302393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:12.304554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.306222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.306671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] PATH_INFO: `/v3/projects/ede438e67af3496cae36fc89abc0aba3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.309264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede438e67af3496cae36fc89abc0aba3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:12.311792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:12.323663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f981e6734744a37a42e6fb761fd6c79'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.324606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f981e6734744a37a42e6fb761fd6c79'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.344193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f981e6734744a37a42e6fb761fd6c79'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.361841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f981e6734744a37a42e6fb761fd6c79'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.362947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec92ece1-a9cf-4e09-8cb0-0b9f0ede4740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f981e6734744a37a42e6fb761fd6c79'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.418562 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2613/5282] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:11 2022] DELETE /identity/v3/users/9f981e6734744a37a42e6fb761fd6c79 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:12.422488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bb0f-f313-4c99-8592-7edd8b0215bd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.424414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bb0f-f313-4c99-8592-7edd8b0215bd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.425454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bb0f-f313-4c99-8592-7edd8b0215bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.449753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede438e67af3496cae36fc89abc0aba3', 'request_id': 'req-2f11e820-66d1-4973-88ba-192d228cd404'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.461527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede438e67af3496cae36fc89abc0aba3', 'request_id': 'req-2f11e820-66d1-4973-88ba-192d228cd404'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.478636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be30bb0f-f313-4c99-8592-7edd8b0215bd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:12.481060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be30bb0f-f313-4c99-8592-7edd8b0215bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.483083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede438e67af3496cae36fc89abc0aba3', 'request_id': 'req-2f11e820-66d1-4973-88ba-192d228cd404'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.512972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] The token cache is being invalidate because project ede438e67af3496cae36fc89abc0aba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:12.515108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.516766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:12.527990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2670/5283] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:12 2022] DELETE /identity/v3/projects/ede438e67af3496cae36fc89abc0aba3 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:12.527990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:12.531286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.585171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:12.610233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:12.668812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2614/5284] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:12.670698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d445a-cfd7-437c-a0c0-c52de47e3be8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.671135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d445a-cfd7-437c-a0c0-c52de47e3be8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.671628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d445a-cfd7-437c-a0c0-c52de47e3be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.703173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be6d445a-cfd7-437c-a0c0-c52de47e3be8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:12.704214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11e820-66d1-4973-88ba-192d228cd404 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:12.704902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be6d445a-cfd7-437c-a0c0-c52de47e3be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.716812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2615/5285] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:12.725765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.726312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.726949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.777141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:12.778358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.800054 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2616/5286] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:12.801369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:12.805001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:12.843440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:12.845269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:12.845554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:12.845888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:12.846558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:12.851768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:12.910665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2671/5287] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:12 2022] POST /identity/v3/projects => generated 329 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:12.911853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:12.959258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e096b9a-cb30-4c0a-96d2-67737556245d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.067893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c642b4-6cee-46b6-8dbb-5b56bc7258de tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.077741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.078552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.079052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.079845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.081704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:13.083246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.085273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.103109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.104243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.104620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.105636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.132130 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2672/5288] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:13.133229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.138218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.139924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.170122 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2617/5289] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:13.173109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.195692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff836-97d1-41dc-bbee-844e8877a493 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.209843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.211464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.211846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.212216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.214902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.220557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.289385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:13.323981 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2618/5290] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:13.328339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.338979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0c448-277d-427b-a678-3e2ce5d76a5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.426177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.427251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.429048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.429567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.431562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.437441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.452901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed6ec7-1a9f-4f2f-980e-1b5037d84685 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.460172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.460792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.461069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.461340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.461972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.464378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.512328 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2619/5291] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:13.513027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2673/5292] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/users => generated 413 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:13.516784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.523707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.529795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:13.545073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea1c9-7905-4b17-9b57-91f613b19ffc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.553702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.554868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.555422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.555642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.555779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e681cd-6881-4976-9679-bcf5d4b3dbcc tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.556391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.558254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.572038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.572877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.573252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.573692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.574451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.576329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.602765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2674/5293] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:13 2022] GET /identity/v3/roles => generated 1654 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:13.603767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.605237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2620/5294] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/roles => generated 243 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:13.609000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.629665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed41be-84f1-4d75-864e-14aed72ce9cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.635025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63693347-c924-4b7c-babb-550044525030 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.639902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.640600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.640950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.641289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.641995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.652729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.653109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.653522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.653913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.654264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.655300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.657597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.676410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2675/5295] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:13 2022] GET /identity/v3/roles => generated 1654 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:13.677908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.691526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2621/5296] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/roles => generated 247 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:13.696506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.699899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38830a-6e4f-4eb0-b6da-b40cbc443535 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.714391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.718591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.719515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.721518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] PATH_INFO: `/v3/projects/e6054fc080d149c7ac0d6f9096d0545d/users/8eced8ee4f5e45b19895743896aae2da/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.757517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48996be-2f6a-4494-8f80-e3ed5024eccf tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.782508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.782508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.782508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.782834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/c3ae9333e59f45cca02efc71b3636164/users/0cf493bd96244cd6abf74680ef19730e/roles/01c54fb7c0d641ba93b5d2f732269a11` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.794234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c3ae9333e59f45cca02efc71b3636164, user_id=0cf493bd96244cd6abf74680ef19730e, role_id=01c54fb7c0d641ba93b5d2f732269a11)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.798773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e051ccb-17f2-4fe6-a297-9ce41c1f4c8c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.800078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6054fc080d149c7ac0d6f9096d0545d, user_id=8eced8ee4f5e45b19895743896aae2da, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.803750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:13.830797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2676/5297] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:13 2022] PUT /identity/v3/projects/e6054fc080d149c7ac0d6f9096d0545d/users/8eced8ee4f5e45b19895743896aae2da/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:13.841425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2622/5298] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:13 2022] PUT /identity/v3/projects/c3ae9333e59f45cca02efc71b3636164/users/0cf493bd96244cd6abf74680ef19730e/roles/01c54fb7c0d641ba93b5d2f732269a11 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:13.844685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:13.845961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.846282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.847766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.924109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] MFA Rules not processed for user `8eced8ee4f5e45b19895743896aae2da`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:13.929176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:13.958218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6b56d-e97c-4211-bca5-d0ae416ffac7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:13.975288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:13.977720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:13.978177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:13.978703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/c3ae9333e59f45cca02efc71b3636164/users/0cf493bd96244cd6abf74680ef19730e/roles/2ddce8e7420040a49d7983ceba399607` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:13.981691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c3ae9333e59f45cca02efc71b3636164, user_id=0cf493bd96244cd6abf74680ef19730e, role_id=2ddce8e7420040a49d7983ceba399607)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:13.986688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.033538 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2677/5299] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:13 2022] PUT /identity/v3/projects/c3ae9333e59f45cca02efc71b3636164/users/0cf493bd96244cd6abf74680ef19730e/roles/2ddce8e7420040a49d7983ceba399607 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:14.034173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:14.041663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.092271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2623/5300] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:14.093267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:14.109263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38b852-24b9-4499-8c6f-27740403dfc7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:14.113074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:14.173478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921de8c3-8f13-4dbc-bcaa-343aca01137e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:14.178922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:14.179612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.179889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.180205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] PATH_INFO: `/v3/projects/05814d65d2844f6b9077b95dabbb5bf3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.181349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05814d65d2844f6b9077b95dabbb5bf3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:14.187295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:14.187929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.188746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.188746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.189280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:14.191138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.195265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.259858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2624/5301] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:14 2022] GET /identity/v3/users => generated 8056 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:14.261635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:14.265890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.284692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05814d65d2844f6b9077b95dabbb5bf3', 'request_id': 'req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.298514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05814d65d2844f6b9077b95dabbb5bf3', 'request_id': 'req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.309424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05814d65d2844f6b9077b95dabbb5bf3', 'request_id': 'req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.338348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] The token cache is being invalidate because project 05814d65d2844f6b9077b95dabbb5bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:14.338685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.339618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b2d9e554-d42e-496d-aa7c-5fe72fd2f716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.341392 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2678/5302] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:14 2022] DELETE /identity/v3/projects/05814d65d2844f6b9077b95dabbb5bf3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:14.342875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710899a-b5bb-468f-8c65-01c1c0316ba7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.343156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710899a-b5bb-468f-8c65-01c1c0316ba7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.343480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710899a-b5bb-468f-8c65-01c1c0316ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.362661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:14.438756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b710899a-b5bb-468f-8c65-01c1c0316ba7 None None] MFA Rules not processed for user `0cf493bd96244cd6abf74680ef19730e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:14.439998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b710899a-b5bb-468f-8c65-01c1c0316ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.467599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5697-e205-4c46-aa00-e69147adaf51 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:14.596535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2679/5303] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:14.597961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.598245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.598542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.619147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:14.619774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.620056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.620568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.626581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.649662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:14.650977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.713714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:14.730880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:14.732954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b4e93-2057-4651-8b31-f28f1103f5c7 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.746689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2625/5304] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:14.746689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc05cd-7c77-451f-a58a-fccf679bf552 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.746689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc05cd-7c77-451f-a58a-fccf679bf552 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.746965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc05cd-7c77-451f-a58a-fccf679bf552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.769328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2680/5305] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:14.770340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:14.777230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.781712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-45fc05cd-7c77-451f-a58a-fccf679bf552 None None] MFA Rules not processed for user `0cf493bd96244cd6abf74680ef19730e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:14.782885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-45fc05cd-7c77-451f-a58a-fccf679bf552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.797854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2626/5306] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3279 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:14.799250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.799676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.799941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.814041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78b4c4-a6b2-4dd2-bc03-d12a08cde5b9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:14.835101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:14.835101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:14.835271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:14.835271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.835271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.836456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] PATH_INFO: `/v3/projects/67dd3aa3a7094a948ad9a8df00dd31a9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.838338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67dd3aa3a7094a948ad9a8df00dd31a9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:14.845089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2627/5307] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:14.845224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.849603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:14.875897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511124cb-4851-4508-a9c9-01b1cd633c72 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:14.887678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:14.890919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:14.891300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:14.891782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:14.892542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:create_trust()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:14.895561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:14.959609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67dd3aa3a7094a948ad9a8df00dd31a9', 'request_id': 'req-a2428db2-05d1-4530-bfba-391398384eb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.966944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67dd3aa3a7094a948ad9a8df00dd31a9', 'request_id': 'req-a2428db2-05d1-4530-bfba-391398384eb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:14.978235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67dd3aa3a7094a948ad9a8df00dd31a9', 'request_id': 'req-a2428db2-05d1-4530-bfba-391398384eb1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.007421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] The token cache is being invalidate because project 67dd3aa3a7094a948ad9a8df00dd31a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:15.008472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.010970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.014610 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2681/5308] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:14 2022] DELETE /identity/v3/projects/67dd3aa3a7094a948ad9a8df00dd31a9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:15.025591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.025591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.027208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:15.089843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2628/5309] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:11:14 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:15.089843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.091666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.110653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84b166-bdb0-4859-b454-b71471190a02 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.191559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428db2-05d1-4530-bfba-391398384eb1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.201149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.202127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.202524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.202882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.204418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:15.204795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.208213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.226199 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2682/5310] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:15.227201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.228935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.235003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.236759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.236759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.236759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] PATH_INFO: `/v3/users/c9a9afc1574f4ba3a61ad7ca46389866` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.238485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a9afc1574f4ba3a61ad7ca46389866)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.244126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.327145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7344-aef9-4d57-a035-36ae7958219b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.331376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.331948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.332345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.332570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.338768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:15.340688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:get_trust(trust_id=fb362b87c51044c78aca20ea0990cb50)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.342255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.348481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a9afc1574f4ba3a61ad7ca46389866'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.351095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a9afc1574f4ba3a61ad7ca46389866'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.355352 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2683/5311] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:15 2022] GET /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50 => generated 833 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:15.355352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.357404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.371483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a9afc1574f4ba3a61ad7ca46389866'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.376265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7c8c6-8145-4b4f-9d5e-3b89afc56a2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.382735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.383527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.383874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.384216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] PATH_INFO: `/v3/users/8eced8ee4f5e45b19895743896aae2da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.386256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eced8ee4f5e45b19895743896aae2da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.388580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.393037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a9afc1574f4ba3a61ad7ca46389866'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.397332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a9afc1574f4ba3a61ad7ca46389866'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.442332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2629/5312] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:15 2022] DELETE /identity/v3/users/c9a9afc1574f4ba3a61ad7ca46389866 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:15.444860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.474671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.489153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eced8ee4f5e45b19895743896aae2da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.491813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eced8ee4f5e45b19895743896aae2da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.512370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eced8ee4f5e45b19895743896aae2da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.538116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eced8ee4f5e45b19895743896aae2da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.540005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eced8ee4f5e45b19895743896aae2da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:15.585874 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2684/5313] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:15 2022] DELETE /identity/v3/users/8eced8ee4f5e45b19895743896aae2da => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:15.587091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.589823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.644200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2eb711-f422-414c-abdc-5055881b7498 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.648840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:15.727750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.728382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.728664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.729068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.729720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.731700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73e288-e224-4d31-9e48-ec4685708c3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.755192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68dc58-b58f-4aaa-a7fb-b63e279f2a4e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.765464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.766182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.766443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.767295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.779800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=fb362b87c51044c78aca20ea0990cb50)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.782606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.787292 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2630/5314] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:15 2022] POST /identity/v3/projects => generated 373 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:15.787292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.787292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.787292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.801337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2685/5315] 10.222.0.52 () {58 vars in 1269 bytes} [Sat Sep 24 21:11:15 2022] GET /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:15.802554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.808539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.827876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:15.829261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbe8d9-22ed-48ab-80ec-7021ba17dc38 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:15.829550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:15.943197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2631/5316] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:15.948430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:15.962383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:15.963663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:15.964166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:15.964676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:15.965950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:15.968790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:15.981925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:16.022446 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2686/5317] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:15 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:16.025159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.027496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.111828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386e5fe-f558-4ba2-b49f-9ecbe5e4ff41 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.123104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972be8-8cbb-4d7f-99f8-2c5caeb2028f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.131721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.132292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.132713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.133062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.133641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.134524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.134918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.135424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/01c54fb7c0d641ba93b5d2f732269a11` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.147519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.150266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:get_role_for_trust(trust_id=fb362b87c51044c78aca20ea0990cb50, role_id=01c54fb7c0d641ba93b5d2f732269a11)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.151877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.152278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.161393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2632/5318] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:15 2022] GET /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/01c54fb7c0d641ba93b5d2f732269a11 => generated 243 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:16.164270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.187486 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2687/5319] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:16.188475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.189811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.203422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2ceaa-b390-4ae2-b230-6c435fcd5e9c tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.209063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.210283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.210473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.211359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.213219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a277-4586-4493-a256-0e34ed27738a service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.219036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.222940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.225764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.226541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.226781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.229934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] PATH_INFO: `/v3/projects/e105facdff49413391f9a3c0993fb0f0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.231333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e105facdff49413391f9a3c0993fb0f0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.233317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.239499 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2633/5320] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:16 2022] GET /identity/v3/roles => generated 1892 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:16.241900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.276763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b066b-4f01-4879-b90d-93a983965afa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.297070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:16.300012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e105facdff49413391f9a3c0993fb0f0', 'request_id': 'req-77e73041-d8fc-4570-88c1-d1455d4193b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.310073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e105facdff49413391f9a3c0993fb0f0', 'request_id': 'req-77e73041-d8fc-4570-88c1-d1455d4193b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.317518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e105facdff49413391f9a3c0993fb0f0', 'request_id': 'req-77e73041-d8fc-4570-88c1-d1455d4193b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.348018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] The token cache is being invalidate because project e105facdff49413391f9a3c0993fb0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:16.348749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.350727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.353607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2688/5321] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:16 2022] DELETE /identity/v3/projects/e105facdff49413391f9a3c0993fb0f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:16.354806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.356377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.377178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.378156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.378522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.378894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/01c54fb7c0d641ba93b5d2f732269a11` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.387367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:get_role_for_trust(trust_id=fb362b87c51044c78aca20ea0990cb50, role_id=01c54fb7c0d641ba93b5d2f732269a11)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.390355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808d84f-d5db-4317-9357-fed9b42731e5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.404434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2634/5322] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:16 2022] HEAD /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/01c54fb7c0d641ba93b5d2f732269a11 => generated 0 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 24 21:11:16.407256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.407650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.408780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.442604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:16.447432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.458744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2635/5323] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:16.460639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.461976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.469612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e73041-d8fc-4570-88c1-d1455d4193b8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.476551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.478146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.478528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.479658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a03c10-5466-4d1e-81ff-58998317ef34 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.479824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.480648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.483532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a2b5f-e6b1-4fc3-a6a5-125aa76c6f2e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.485160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.486028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.486963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.487556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/2ddce8e7420040a49d7983ceba399607` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.497318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:get_role_for_trust(trust_id=fb362b87c51044c78aca20ea0990cb50, role_id=2ddce8e7420040a49d7983ceba399607)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.499469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.507020 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2689/5324] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:16 2022] GET /identity/v3/roles => generated 1892 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:16.508454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.510773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.511082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Could not find role: 2ddce8e7420040a49d7983ceba399607.: keystone.exception.RoleNotFound: Could not find role: 2ddce8e7420040a49d7983ceba399607. Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:16.514925 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2ddce8e7420040a49d7983ceba399607. Sep 24 21:11:16.516031 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:16.516889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2636/5325] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:16 2022] GET /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/2ddce8e7420040a49d7983ceba399607 => generated 110 bytes in 57 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:16.517569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.535154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6413604-556a-4ffe-899c-7334eb714f2e tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.543352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.543770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:16.544069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.544352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.544624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] PATH_INFO: `/v3/projects/0340f96a146d4a3bb746713ae6ae6ebf/users/c165554c58674e57a655606a401a0f3b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.545241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.564317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2690/5326] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:16.565938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.566569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0340f96a146d4a3bb746713ae6ae6ebf, user_id=c165554c58674e57a655606a401a0f3b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.571519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.575447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:16.580775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb36d54-306d-4908-bbf0-597e3d8057cc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.589139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.589860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.590306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.590709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/2ddce8e7420040a49d7983ceba399607` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.595902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2637/5327] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:16 2022] PUT /identity/v3/projects/0340f96a146d4a3bb746713ae6ae6ebf/users/c165554c58674e57a655606a401a0f3b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:16.596745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.608485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:get_role_for_trust(trust_id=fb362b87c51044c78aca20ea0990cb50, role_id=2ddce8e7420040a49d7983ceba399607)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.611461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-66ea1fe2-cdce-45f1-8924-7de7c24b5772 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Could not find role: 2ddce8e7420040a49d7983ceba399607.: keystone.exception.RoleNotFound: Could not find role: 2ddce8e7420040a49d7983ceba399607. Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:16.626011 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2ddce8e7420040a49d7983ceba399607. Sep 24 21:11:16.627144 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:16.630576 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2691/5328] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:16 2022] HEAD /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50/roles/2ddce8e7420040a49d7983ceba399607 => generated 0 bytes in 63 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 24 21:11:16.633563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.634547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.634964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.681644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] MFA Rules not processed for user `c165554c58674e57a655606a401a0f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:16.682896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.728234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfc65f-0646-42d0-baa2-903f5827f6de admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.738754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.739391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.739670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.740212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.742292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:16.742607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.744594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.761141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2638/5329] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:16.762146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.793413 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2692/5330] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:16.794452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.813075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5240ad-3614-4719-b987-33e0acd66019 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.821009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.821642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.821918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.822185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.822936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.825399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.848252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0889b2-8111-493a-9583-0caa740d2785 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:16.853535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:16.854525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:16.855034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:16.855520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:16.858191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:16.866523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorizing `identity:delete_trust(trust_id=fb362b87c51044c78aca20ea0990cb50)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:16.868717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:16.891720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'fb362b87c51044c78aca20ea0990cb50', 'request_id': 'req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.892826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'fb362b87c51044c78aca20ea0990cb50', 'request_id': 'req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.896082 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2693/5331] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:16.897851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.901658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.903725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'fb362b87c51044c78aca20ea0990cb50', 'request_id': 'req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:16.916479 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2639/5332] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:11:16 2022] DELETE /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:16.916479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:16.916479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:16.933715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fa017-fa1b-4fce-ab8f-c56e51e20ae5 tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.012277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.013065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.013446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.013896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.014589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.017318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.056373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2640/5333] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:17.057477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.059507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:17.072792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418907c-0351-4754-b25d-62de989fa8c9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.093217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.095895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.097107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.098460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.101383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.105539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.178116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2694/5334] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:16 2022] POST /identity/v3/projects => generated 371 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:17.179696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.192487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc9d-150c-4602-8c61-a44ef7f86ac5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.316250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:17.358721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf493bd96244cd6abf74680ef19730e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3ae9333e59f45cca02efc71b3636164', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-2002754951', 'tempest-DelegatedRole-610326847'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.359942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.360392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.360836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] PATH_INFO: `/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Could not find trust: fb362b87c51044c78aca20ea0990cb50.: keystone.exception.TrustNotFound: Could not find trust: fb362b87c51044c78aca20ea0990cb50. Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 24 21:11:17.371156 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: fb362b87c51044c78aca20ea0990cb50. Sep 24 21:11:17.372395 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:17.373353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf7471-c385-422e-80aa-c0dd1cb440ed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.375113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2641/5335] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:17 2022] GET /identity/v3/OS-TRUST/trusts/fb362b87c51044c78aca20ea0990cb50 => generated 111 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:17.376127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.386193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.388092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.390004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.390303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.391073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.392182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903790-38c3-4c8d-9804-5e43cb6c55bd tempest-TrustsV3TestJSON-1340014609 tempest-user-882130252] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.393412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.398613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.399709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.400175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.400717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.402199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.404577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.417735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:UPDATE=1,keystone:INSERT=6,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:17.445215 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2642/5336] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:17 2022] POST /identity/v3/users => generated 292 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:17.446002 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2695/5337] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:17 2022] GET /identity/v3/roles => generated 1892 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:17.446620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.446894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.448740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:17.449108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:17.471625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8ab2b-96fc-4ab4-97f4-a74d674127e5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.488186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.488988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.489348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.489648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] PATH_INFO: `/v3/projects/e6054fc080d149c7ac0d6f9096d0545d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.492092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6054fc080d149c7ac0d6f9096d0545d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.496197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.595945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4d0f-406a-4333-892f-22789a50446f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.634830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6054fc080d149c7ac0d6f9096d0545d', 'request_id': 'req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.642002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6054fc080d149c7ac0d6f9096d0545d', 'request_id': 'req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.654186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6054fc080d149c7ac0d6f9096d0545d', 'request_id': 'req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.689170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] The token cache is being invalidate because project e6054fc080d149c7ac0d6f9096d0545d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:17.689489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.690420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.695264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2696/5338] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:17 2022] DELETE /identity/v3/projects/e6054fc080d149c7ac0d6f9096d0545d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:17.695264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.697957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:17.711699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.712340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.712793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.713289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/2ddce8e7420040a49d7983ceba399607` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.716330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2ddce8e7420040a49d7983ceba399607)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.716330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.799482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role 2ddce8e7420040a49d7983ceba399607 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:17.800131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.800883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:17.805908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2643/5339] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:17 2022] DELETE /identity/v3/roles/2ddce8e7420040a49d7983ceba399607 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:17.807091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:17.808523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:17.820861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b0d60-8dc7-495e-a55e-e0fa757b93f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.941053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f143d-995d-4733-ba39-e5f8954a3e27 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:17.978599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.983287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.983287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.983287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.983287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:17.986754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9448068-e44e-47ff-917f-8a9df9dcb048 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:17.995546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:17.996732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:17.997847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:17.998478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:17.999768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.001764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.010425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2644/5340] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:17 2022] GET /identity/v3/roles => generated 1654 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:18.011748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.012150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.012534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.017452 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2697/5341] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:17 2022] GET /identity/v3/roles => generated 1654 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:18.018737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.022062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:18.060904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.067740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:18.109772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:18.120426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.148393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2645/5342] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:18.148393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.158028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4219815-7420-429e-93c0-ef5252ca40a8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:18.164296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:18.165151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.165514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.165885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] PATH_INFO: `/v3/projects/7a9da6d6af7a4fb7b1133acb9082b57c/users/ed97eb21296641cda0459702b8090804/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.183946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3229e-dda1-40b8-954d-23a81a382c7f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:18.195842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:18.196785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.203540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9da6d6af7a4fb7b1133acb9082b57c, user_id=ed97eb21296641cda0459702b8090804, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.203971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.204532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/01c54fb7c0d641ba93b5d2f732269a11` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.206274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=01c54fb7c0d641ba93b5d2f732269a11)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.210668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.210884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.248511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2646/5343] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:18 2022] PUT /identity/v3/projects/7a9da6d6af7a4fb7b1133acb9082b57c/users/ed97eb21296641cda0459702b8090804/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:18.249449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.254252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.319669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role 01c54fb7c0d641ba93b5d2f732269a11 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:18.323823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.325136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2d7b959-5133-40ee-ba5e-8f94f33cf713 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.327587 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2698/5344] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:18 2022] DELETE /identity/v3/roles/01c54fb7c0d641ba93b5d2f732269a11 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:18.329252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d2bc6-4e14-4872-b159-68a6acf3a815 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.330542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d2bc6-4e14-4872-b159-68a6acf3a815 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.331186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d2bc6-4e14-4872-b159-68a6acf3a815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.364147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4a9d2bc6-4e14-4872-b159-68a6acf3a815 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:18.366149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4a9d2bc6-4e14-4872-b159-68a6acf3a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.446969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495145c8-d8b1-4e8a-9acb-38d8a1d1045f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:18.523261 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2699/5345] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:18.524573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.524967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.525113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.569045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:18.570262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.570767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.571201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] PATH_INFO: `/v3/projects/f39f22cf5bb04332a9c8579ff38d859c/users/0ada02cf79d6453b90cd6ed4686dced7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.576798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] MFA Rules not processed for user `ed97eb21296641cda0459702b8090804`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:18.578062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.589723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39f22cf5bb04332a9c8579ff38d859c, user_id=0ada02cf79d6453b90cd6ed4686dced7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.593447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.623108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2647/5346] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:18 2022] PUT /identity/v3/projects/f39f22cf5bb04332a9c8579ff38d859c/users/0ada02cf79d6453b90cd6ed4686dced7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:18.624507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.628678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.755415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:18.778943 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2700/5347] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:18.780737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.795721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678101-31fb-443a-b22c-ca632618735b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:18.796435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2984d-e774-408e-9a8a-fdf1f215926b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:18.803726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:18.804380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.804663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.804939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/0cf493bd96244cd6abf74680ef19730e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.807557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=0cf493bd96244cd6abf74680ef19730e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.810115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.813485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:18.903612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf493bd96244cd6abf74680ef19730e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.904712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf493bd96244cd6abf74680ef19730e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.909870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:18.912805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:18.913095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:18.913372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:18.914040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:18.914330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:18.916006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:18.930289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf493bd96244cd6abf74680ef19730e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.939649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf493bd96244cd6abf74680ef19730e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.941303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf493bd96244cd6abf74680ef19730e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:18.948908 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2701/5348] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:18.948908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.955638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.989754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2648/5349] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:18 2022] DELETE /identity/v3/users/0cf493bd96244cd6abf74680ef19730e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:18.991852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:18.994028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:18.998348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c44a-1725-4811-baf9-3d4e3822b295 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.107625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.109241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.109520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.109920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.112406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.113771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.122284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b0e79-7e1a-42a1-b9df-566ad296386e tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.128058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.131591 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2702/5350] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:19.131591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:19.133148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.133936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.134445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:19.134674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.138405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:19.156073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b3bb3-168b-4f01-ac80-c2b5a7c527ec admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.161665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.163610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.163907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.164184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.165024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.173131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.289297 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2703/5351] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:19 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:19.289297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:19.291368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:19.297370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.299527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.330039 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2649/5352] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:19.331950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:19.360529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ba7d6-77d5-4d74-901b-b2e248d82496 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.416078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:19.432384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1373d6f-abdd-4754-9ea2-f32076fe71f0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.439457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.440394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.440686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.440968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/c3ae9333e59f45cca02efc71b3636164` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.443240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c3ae9333e59f45cca02efc71b3636164)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.445590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.457212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.459375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.459899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.460380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] PATH_INFO: `/v3/projects/f39f22cf5bb04332a9c8579ff38d859c/users/0ada02cf79d6453b90cd6ed4686dced7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.460640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:19.463728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39f22cf5bb04332a9c8579ff38d859c, user_id=0ada02cf79d6453b90cd6ed4686dced7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.468317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.486593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2650/5353] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:19 2022] PUT /identity/v3/projects/f39f22cf5bb04332a9c8579ff38d859c/users/0ada02cf79d6453b90cd6ed4686dced7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:19.487564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:19.585154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ae9333e59f45cca02efc71b3636164', 'request_id': 'req-951ec212-640e-4618-8c57-0100a1027fe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.592150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ae9333e59f45cca02efc71b3636164', 'request_id': 'req-951ec212-640e-4618-8c57-0100a1027fe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.604963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ae9333e59f45cca02efc71b3636164', 'request_id': 'req-951ec212-640e-4618-8c57-0100a1027fe1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.654466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project c3ae9333e59f45cca02efc71b3636164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:19.655970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.656883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.658967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2704/5354] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:19 2022] DELETE /identity/v3/projects/c3ae9333e59f45cca02efc71b3636164 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:19.660022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:19.668270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:19.683553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe1a27-8200-4dc5-abc8-adcee89a606b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.804776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec212-640e-4618-8c57-0100a1027fe1 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:19.811886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.813337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.813835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.814168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.815428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.825862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:19.830101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.830479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.830864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] PATH_INFO: `/v3/users/ed97eb21296641cda0459702b8090804` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.832388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79f4e2-dec7-4cb6-9285-142332a5be13 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.832530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed97eb21296641cda0459702b8090804)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:19.837843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:19.890827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2651/5355] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:19 2022] POST /identity/v3/users => generated 293 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:19.892450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047814-205a-4809-bf57-3322422a6e98 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:19.892923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047814-205a-4809-bf57-3322422a6e98 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:19.902602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96047814-205a-4809-bf57-3322422a6e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:19.939346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed97eb21296641cda0459702b8090804'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.940142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed97eb21296641cda0459702b8090804'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.966211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed97eb21296641cda0459702b8090804'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:19.980223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-96047814-205a-4809-bf57-3322422a6e98 None None] MFA Rules not processed for user `0ada02cf79d6453b90cd6ed4686dced7`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:19.983838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-96047814-205a-4809-bf57-3322422a6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:20.005207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed97eb21296641cda0459702b8090804'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:20.008363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed97eb21296641cda0459702b8090804'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:20.062141 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2705/5356] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:19 2022] DELETE /identity/v3/users/ed97eb21296641cda0459702b8090804 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:20.062141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:20.067361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:20.120493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:20.158073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:20.213421 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2652/5357] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:20.214806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96047814-205a-4809-bf57-3322422a6e98 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:20.220507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-96047814-205a-4809-bf57-3322422a6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:20.257367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96047814-205a-4809-bf57-3322422a6e98 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:20.265144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558c5d1-1a23-4263-8235-e30c5413825c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:20.278369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:20.279291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:20.279578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:20.279985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:20.281431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:20.284622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2e37-950f-496e-8a63-b7ed11ebb35c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:20.397252 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2706/5358] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:20 2022] POST /identity/v3/projects => generated 329 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:20.423452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:20.424503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:20.424805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:20.432345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:20.432913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:20.433960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:20.434248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:20.435626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:20.437290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:20.477640 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2653/5359] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:20.477640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:20.547704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:20.547704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:20.701649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d596d15-e8d4-46f5-9cfc-3d3a13bd5f2b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:20.850665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:20.854132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2707/5360] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:20.854132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:20.855333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:20.856027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:20.857112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:20.858032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:20.859917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:20.902110 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2654/5361] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:11:20 2022] POST /identity/v3/domains => generated 275 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:20.903375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:20.956088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:21.019995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f21d-24e2-498b-9cb2-bc172f9d888d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.025918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb652c-ba31-4bf2-b9c6-250ea0ae71ae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.049293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.049994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.050279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.051512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.052175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.057119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.092550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2655/5362] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:21.098074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.102443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:21.175663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:21.211664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.212296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.212579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.212856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.213513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.215645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.252147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc072fb-e676-4d8e-b65f-60be3ce6e640 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.268430 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2708/5363] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:20 2022] POST /identity/v3/users => generated 413 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:21.268430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.274837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.277573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.279001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.279329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.311400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.317634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.340336 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2656/5364] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:21.341500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.356228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065468e-7a11-46fe-a84f-bd9962d09696 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.363944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.365398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.365684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.366037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] PATH_INFO: `/v3/projects/5fa863cc66874d03889b96d25a37e606/users/3af31013bd244b14ac9bf47b7d9fe392/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.376063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1645ac-f587-4115-8163-9e68ee5d28b3 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.382559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.383559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.384002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.384423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.385151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.387344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.387665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa863cc66874d03889b96d25a37e606, user_id=3af31013bd244b14ac9bf47b7d9fe392, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.390983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e31582-5230-4459-8bb5-52bb84736aa5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.415924 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2657/5365] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:21 2022] PUT /identity/v3/projects/5fa863cc66874d03889b96d25a37e606/users/3af31013bd244b14ac9bf47b7d9fe392/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:21.420317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.422183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.422470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.452432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2709/5366] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/projects => generated 402 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:21.453095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.456145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:21.457585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:21.564261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113a7b-e7d6-4118-9577-bc6353b49125 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.572822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.573616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.573907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.574373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.575387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.577486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c2afe-bb42-4106-ae94-9490b51de54a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.598963 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2710/5367] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/roles => generated 243 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:21.600643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.600943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.602147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.656558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] MFA Rules not processed for user `3af31013bd244b14ac9bf47b7d9fe392`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:21.657950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:21.660951 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2658/5368] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:21.665613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.740387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:21.744028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:21.777656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8eb0e-c7f4-48d3-b930-e649975bbfcf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.783900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.784626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.784963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.785242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.785872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.788467 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2711/5369] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:21.789209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.789421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.813268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a9d1-d6e9-4790-8b34-f27d26c3b3bf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.814974 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2659/5370] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/groups => generated 301 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:21.816016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.820202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:21.825926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.826605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.826890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.827160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.827822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.829426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.864119 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2712/5371] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/roles => generated 246 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:21.864119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:21.937876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679a6e-89b9-4797-8ff3-52c6e347fc4b tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:21.945480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:21.947272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:21.947272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:21.947272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:21.959729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:21.962433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:21.986731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2660/5372] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:11:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:11:21.988048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.005703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e11c5-1e56-443b-91dc-2d7ad97a7d9a service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.022870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.023766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.024207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.024642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.027102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.027499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4356f2-88b0-4cc0-8b58-538aaf5a7375 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.031064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.033406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.034097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.034448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.034858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.035565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.039842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.092753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2661/5373] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/users => generated 442 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:22.092753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.126157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2713/5374] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:21 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:22.127615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.133253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:22.136124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044251-4d42-4ed0-8ede-6cca298f6f1e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.289919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.291253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.292365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.292839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/064fd4abcc7147859312b8eaaffa8cf9/users/bdaff246971c42d8bfb0526a4912b2bb/roles/766596adfe3e424ca85983fe765756a8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.294958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fe964-0378-40b4-84e9-5df98a7fea49 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.323835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.324772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.325280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.325870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] PATH_INFO: `/v3/users/a79ff8f5c7554b60b606f779b7556f18` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.328992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79ff8f5c7554b60b606f779b7556f18)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.331174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.367093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=064fd4abcc7147859312b8eaaffa8cf9, user_id=bdaff246971c42d8bfb0526a4912b2bb, role_id=766596adfe3e424ca85983fe765756a8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.372642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.402893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2662/5375] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:22 2022] PUT /identity/v3/projects/064fd4abcc7147859312b8eaaffa8cf9/users/bdaff246971c42d8bfb0526a4912b2bb/roles/766596adfe3e424ca85983fe765756a8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:22.404701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.417943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:22.476768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79ff8f5c7554b60b606f779b7556f18'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.477552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79ff8f5c7554b60b606f779b7556f18'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.481700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:22.492814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79ff8f5c7554b60b606f779b7556f18'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.502361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:22.528425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79ff8f5c7554b60b606f779b7556f18'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.533003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79ff8f5c7554b60b606f779b7556f18'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.566777 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2714/5376] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:22 2022] DELETE /identity/v3/users/a79ff8f5c7554b60b606f779b7556f18 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:22.568547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.574421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:22.624618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae354060-a395-4ca3-9efc-17813a6882c2 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.762441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.763186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.763358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.763768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] PATH_INFO: `/v3/projects/7a9da6d6af7a4fb7b1133acb9082b57c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.765739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9da6d6af7a4fb7b1133acb9082b57c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.767831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.798847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74333cc-b28b-48f2-b6e1-a7284dbfa3bd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.811366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.812756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.814209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.814926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:22.817471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:22.820340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:22.852436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2715/5377] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:22 2022] POST /identity/v3/roles => generated 239 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:22.855798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.859284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:22.866814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9da6d6af7a4fb7b1133acb9082b57c', 'request_id': 'req-c8db20b3-ffef-48b9-a052-9978674f3173'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.885291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9da6d6af7a4fb7b1133acb9082b57c', 'request_id': 'req-c8db20b3-ffef-48b9-a052-9978674f3173'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.891813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3718534-47e0-49cb-aa63-3ebf49f21d6e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:22.903869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9da6d6af7a4fb7b1133acb9082b57c', 'request_id': 'req-c8db20b3-ffef-48b9-a052-9978674f3173'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.952779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] The token cache is being invalidate because project 7a9da6d6af7a4fb7b1133acb9082b57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:22.953086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.954946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:22.961771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2663/5378] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:22 2022] DELETE /identity/v3/projects/7a9da6d6af7a4fb7b1133acb9082b57c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:22.962725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:22.966200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:22.998204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:22.999160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:22.999520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:22.999864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.000635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.004858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cbda5-252e-43bc-b293-927236809afe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.040894 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2716/5379] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:22 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:23.042455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.042870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.043702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.076144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:23.077946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.094629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2717/5380] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:23.100231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.104586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.121341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:23.129576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db20b3-ffef-48b9-a052-9978674f3173 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.137495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.138975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.140121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.140986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/064fd4abcc7147859312b8eaaffa8cf9/users/bdaff246971c42d8bfb0526a4912b2bb/roles/7f470358b2344d0abfb5ec4359a59660` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.144340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:create_grant(project_id=064fd4abcc7147859312b8eaaffa8cf9, user_id=bdaff246971c42d8bfb0526a4912b2bb, role_id=7f470358b2344d0abfb5ec4359a59660)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.148867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cc922b-455f-472a-854c-b09ebe719908 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.183666 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2664/5381] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:22 2022] PUT /identity/v3/projects/064fd4abcc7147859312b8eaaffa8cf9/users/bdaff246971c42d8bfb0526a4912b2bb/roles/7f470358b2344d0abfb5ec4359a59660 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:23.185023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.186782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.187613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.229912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:23.234221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.282712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:23.320256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a78791-20bd-44bf-a7a0-e39ff1b23c73 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.387017 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2665/5382] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:23.387987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.389365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.407979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c6ac0-3405-4ee8-ae1d-335f2b7f62a6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.415704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.416508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.417062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.417349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.418195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.421099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.443994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2666/5383] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:23 2022] GET /identity/v3/roles => generated 2121 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:23.443994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.444768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.478529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.479279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.479555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.479837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.486550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=11a638fbbbca49e681b6a9db4c5f9f5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.489352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.510582 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2718/5384] 10.222.0.52 () {60 vars in 1585 bytes} [Sat Sep 24 21:11:23 2022] PUT /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:23.511378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.617726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29054838-e5d1-4f04-ad3f-46143eeb6302 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.698154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26921a75-8293-4f0a-9499-fc3d6d8fae97 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.727391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.728566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.729587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.730380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.731416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.734246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ae56-a004-4f3b-8e72-73b1bdba8c6b tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.756052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.757511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.758054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.758330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.811133 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2719/5385] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:23 2022] GET /identity/v3/users => generated 8790 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:23.820788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.821516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.822003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:23.851833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:23.913944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:23.925507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:23.925910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:23.928212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:23.948023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2667/5386] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:23.948979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.963808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62eef-01d0-43e8-85e7-dbfd4b115efe service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:23.964548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2720/5387] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:23.970749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:23.982436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:23.983157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:23.983541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:23.983828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] PATH_INFO: `/v3/projects/13f013dc8b174609bc320320e3b034d0/users/ce802418add84d59ab8a256ae1ea2e1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.007952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f013dc8b174609bc320320e3b034d0, user_id=ce802418add84d59ab8a256ae1ea2e1e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.016731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b319e-f1b3-4813-9ae6-3756bec44006 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.059021 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2668/5388] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:23 2022] PUT /identity/v3/projects/13f013dc8b174609bc320320e3b034d0/users/ce802418add84d59ab8a256ae1ea2e1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.059021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.061493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.061879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.166370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] MFA Rules not processed for user `bdaff246971c42d8bfb0526a4912b2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:24.169006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.172138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceee0b-74e6-458e-8d9a-e8ef74132e3c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.210010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:24.325166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.325870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.326153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.326429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.328382 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2669/5389] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:24.330086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:check_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=11a638fbbbca49e681b6a9db4c5f9f5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.331354 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:24.331354 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:24.331354 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:24.333382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.343763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.351541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2721/5390] 10.222.0.52 () {58 vars in 1567 bytes} [Sat Sep 24 21:11:23 2022] HEAD /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.352591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.450151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77a7e5-8dd7-417b-b62f-82e5561fd34a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.455463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ab8c7-1dea-45b8-9672-43cce6eb35e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.455953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.456640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.456993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.457339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.458066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:24.459256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.460716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.461549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.462305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.462619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.463037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.463963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.465611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279878e-7507-48e5-9d1f-4b05672eff98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.476423 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2670/5391] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:24.477451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.483352 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2722/5392] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:24 2022] GET /identity/v3/roles => generated 2121 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:24.484285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.484629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.484973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.486178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.502533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddcb93-b4f4-4211-9e15-df70dd095fb6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.510967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.511669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.511989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.512271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] PATH_INFO: `/v3/projects/0f0bd29c38f24ebfbf4a66462adac801` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.513762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0bd29c38f24ebfbf4a66462adac801)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.516766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.519285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] MFA Rules not processed for user `bdaff246971c42d8bfb0526a4912b2bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:24.521813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.529971 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2723/5393] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:24.532650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.557827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea282fb-049a-4581-84de-2ef22f42a36d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.564064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.565269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.565745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.566248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.570192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=11a638fbbbca49e681b6a9db4c5f9f5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.571532 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:24.571532 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:24.571532 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:24.573575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.579844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:24.580786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 11a638fbbbca49e681b6a9db4c5f9f5a was removed from group ffb3cf6be1374d0596cf363525a6c24a on project 5376a1c6d7e145b1aca1cf1b936d0acf. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:24.581070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.581867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.603686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0bd29c38f24ebfbf4a66462adac801', 'request_id': 'req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.607019 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2724/5394] 10.222.0.52 () {58 vars in 1569 bytes} [Sat Sep 24 21:11:24 2022] DELETE /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/11a638fbbbca49e681b6a9db4c5f9f5a/inherited_to_projects => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.608562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.610080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.617974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0bd29c38f24ebfbf4a66462adac801', 'request_id': 'req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.634551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0bd29c38f24ebfbf4a66462adac801', 'request_id': 'req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.658605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] The token cache is being invalidate because project 0f0bd29c38f24ebfbf4a66462adac801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:24.659073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.661111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.663747 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2671/5395] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:24 2022] DELETE /identity/v3/projects/0f0bd29c38f24ebfbf4a66462adac801 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.665041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.666519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.745814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5854a-3bf0-4500-8735-bb67a8b40da6 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.753073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.753997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.754527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.755010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.756732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.758964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2cda2-8cfd-4410-a8e7-376bdbe8871b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.759312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.768849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.769545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.770081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.770364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] PATH_INFO: `/v3/projects/13f013dc8b174609bc320320e3b034d0/users/ce802418add84d59ab8a256ae1ea2e1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.774440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f013dc8b174609bc320320e3b034d0, user_id=ce802418add84d59ab8a256ae1ea2e1e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.777328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.799053 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2672/5396] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:24 2022] PUT /identity/v3/projects/13f013dc8b174609bc320320e3b034d0/users/ce802418add84d59ab8a256ae1ea2e1e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.801930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.804691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.804933 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2725/5397] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:24 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:24.806082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:24.808316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:24.826779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:24.891032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec219096-ed8e-411e-a2a3-2b3b9bff523f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.916516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d23a-eec1-44dc-b5d8-69f31f7da5a7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:24.933081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.933732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.934012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.934271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/11a638fbbbca49e681b6a9db4c5f9f5a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.935715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=11a638fbbbca49e681b6a9db4c5f9f5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.937840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.985620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 11a638fbbbca49e681b6a9db4c5f9f5a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:24.986669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.986840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:24.987517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.987849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.988007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5829da30-2b11-4c0a-b37c-89d875f697d5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:24.988111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:24.988832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:create_trust()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:24.990998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746cc47-caad-4f06-b9a7-66f78fcef932 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:24.991767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2726/5398] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:24 2022] DELETE /identity/v3/roles/11a638fbbbca49e681b6a9db4c5f9f5a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:24.993531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:24.993835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:24.994047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.022914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:25.038300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:25.040137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.125550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2673/5399] 10.222.0.52 () {60 vars in 1174 bytes} [Sat Sep 24 21:11:24 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:25.125550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.126125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.126725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.153030 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2727/5400] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:25.154270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.156072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.174649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17294bfd-573f-49b4-a904-455b6767a103 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.183486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] MFA Rules not processed for user `ce802418add84d59ab8a256ae1ea2e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:25.184867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.186488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.187613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.188057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.189739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.190392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.192913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.222051 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2728/5401] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/users => generated 293 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:25.223072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.225654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.298057 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2674/5402] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:25.298057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.300107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.329207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13e36-2cd2-4f76-9891-7af19d1a9580 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.337328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.337951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.338230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.338521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.339191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.341456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379872-3dcc-4985-bae7-5c5688d7f420 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.348596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:25.409183 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2729/5403] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/roles => generated 239 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:25.410874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.411323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.413202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.445246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ec1ea-1e35-491d-94e5-8e3ea31e9343 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.453995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.456837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.457877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.458242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.471941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:25.473197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:25.476389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:get_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.478166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.494821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2730/5404] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:25.495787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.496761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2675/5405] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1 => generated 834 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:25.497531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.508191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d79009-8d03-41ab-895d-55aa9f5f5fd3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.514010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.514806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.515089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.515361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.516231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.517985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.526958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2dbb1e-6c26-4f7d-99be-99657fc22a56 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.533428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.534065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.534327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.534633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.535260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.538439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.555560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2731/5406] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/roles => generated 2121 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:25.556545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.571100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05259d12-2314-4dec-9215-bdde6ac8880d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.589548 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2676/5407] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/services => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:25.590897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.618708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.619419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.619760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.620048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.623557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28a6b-b3e6-4b5c-8feb-20cf9ae28527 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.629988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.631809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.632108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.632431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.633088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:25.633387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.649416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.651886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:25.675834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2677/5408] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:25.675834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.688073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=cdffa045f237455da96099c2b4766a59)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.688073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.709041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5debdf4-e239-48a5-bb50-a93ae1293a23 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.715256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.716038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.716408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.716779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.726399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.728194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.733555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2732/5409] 10.222.0.52 () {60 vars in 1582 bytes} [Sat Sep 24 21:11:25 2022] PUT /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:25.734565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.741940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2678/5410] 10.222.0.52 () {58 vars in 1269 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles => generated 370 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:25.742941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.778092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433f0f9-4f0d-46ee-8565-7adf21119ab6 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.819497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc84cd2-023a-40cf-b31b-b391bb0cd723 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.824879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.825635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.825911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.826229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.826871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.828479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.841811 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2733/5411] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/roles => generated 2121 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:25.842787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.855357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132d316-f932-4402-827f-c99e4fa93346 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:25.862214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.862588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.862957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.863324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.864050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.867858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.891606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:25.895639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:25.896344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:25.897410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:25.898718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:25.900676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: 'False' Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise error Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'False' Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.911162 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:11:25.917782 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'False' Sep 24 21:11:25.918980 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:25.920273 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2679/5412] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/endpoints => generated 283 bytes in 178 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:11:25.921245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:25.926163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2734/5413] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:25 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:25.927976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.031214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1cb08-52cc-4dc3-90cc-e96703da34a7 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.111170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ede988-8323-4555-8fd3-25e5894f6056 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.111170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.111170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.111537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.111537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/766596adfe3e424ca85983fe765756a8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.111537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:get_role_for_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1, role_id=766596adfe3e424ca85983fe765756a8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.111537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.122895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2735/5414] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:25 2022] GET /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/766596adfe3e424ca85983fe765756a8 => generated 243 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:26.123780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.137705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.139539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.140006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.140455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.143249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:check_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=cdffa045f237455da96099c2b4766a59)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.148031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:26.157426 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:26.157426 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:26.157426 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:26.161716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.188055 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2680/5415] 10.222.0.52 () {58 vars in 1564 bytes} [Sat Sep 24 21:11:25 2022] HEAD /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:26.188055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.207760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:26.252950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d9313-1afc-45cc-965f-e61605d5bd66 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.264540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.265215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.265532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.266254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] PATH_INFO: `/v3/projects/cab8f6881ee24233aa25f3df3f2876e1/users/27a03e9e5f7c43b4b6ef9433cf868ff1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.296232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab8f6881ee24233aa25f3df3f2876e1, user_id=27a03e9e5f7c43b4b6ef9433cf868ff1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.299302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.332634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee644f-20b1-4184-a3e9-95c42c8c5175 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.338600 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2736/5416] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:26 2022] PUT /identity/v3/projects/cab8f6881ee24233aa25f3df3f2876e1/users/27a03e9e5f7c43b4b6ef9433cf868ff1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:26.339539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.463698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.464322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.464599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.470524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.470524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.470524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.479928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c97ea-68ee-408a-93ba-a3b499a5293c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: 'True' Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise error Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'True' Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.481454 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:11:26.483128 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'True' Sep 24 21:11:26.484246 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:26.491620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2681/5417] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:11:26 2022] POST /identity/v3/endpoints => generated 281 bytes in 305 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:11:26.492108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.492588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.494177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.494720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.495256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.496426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.500204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.534109 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2737/5418] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:26 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:26.535334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.597470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388da32-77a7-4f48-81ab-f1f9b24a88a3 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.602133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.607167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.607448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.607831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/766596adfe3e424ca85983fe765756a8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.616929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:get_role_for_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1, role_id=766596adfe3e424ca85983fe765756a8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.618541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee13a4-ee47-497d-aa3d-2b5dc390bd2b tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.640930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2682/5419] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:26 2022] HEAD /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/766596adfe3e424ca85983fe765756a8 => generated 0 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) Sep 24 21:11:26.640930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.640930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.640930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.663227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe525f-0337-4958-9a92-d99b21f9cb42 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.674823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.675528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.675813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.676144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.687298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] MFA Rules not processed for user `27a03e9e5f7c43b4b6ef9433cf868ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:26.688596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:26.705197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=cdffa045f237455da96099c2b4766a59)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.715725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.728545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role cdffa045f237455da96099c2b4766a59 was removed from user a592b7077dc3419ea466c6d39fbc3fc0 on project 5376a1c6d7e145b1aca1cf1b936d0acf. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:26.729732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:26.731059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:26.768013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2738/5420] 10.222.0.52 () {58 vars in 1566 bytes} [Sat Sep 24 21:11:26 2022] DELETE /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/cdffa045f237455da96099c2b4766a59/inherited_to_projects => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:26.768013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.768509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:26.803454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:26.812227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:26.864393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2683/5421] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3396 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:26.864393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.866573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:26.889055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f84c07-df0f-479c-99a3-c5355b8ef094 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.909021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebac45-855d-4c62-8cd7-6f05af515ee0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:26.919789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.920784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.923057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.929818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.930878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.933375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.960190 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1478332577. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:11:26.960190 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:11:26.960190 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:26.965314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:26.966556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:26.966556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:26.966556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:26.967313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:26.969357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:26.982197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2684/5422] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:26 2022] GET /identity/v3/roles => generated 2121 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:26.987147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:26.989285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.027143 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2739/5423] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:11:26 2022] POST /identity/v3/endpoints => generated 360 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:27.030446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.034763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.104001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6048-045a-454b-81c0-1911816cca11 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.138985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.139614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.139905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.140207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/7f470358b2344d0abfb5ec4359a59660` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.150376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:get_role_for_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1, role_id=7f470358b2344d0abfb5ec4359a59660)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.164960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Could not find role: 7f470358b2344d0abfb5ec4359a59660.: keystone.exception.RoleNotFound: Could not find role: 7f470358b2344d0abfb5ec4359a59660. Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:27.188738 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 7f470358b2344d0abfb5ec4359a59660. Sep 24 21:11:27.189872 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:27.195890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2685/5424] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:11:26 2022] GET /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/7f470358b2344d0abfb5ec4359a59660 => generated 110 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:27.196803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.202560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.224928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97adf-b90a-4433-9969-ec5ed84073b8 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.241996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.242689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.242976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.243252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/cdffa045f237455da96099c2b4766a59` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.246226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=cdffa045f237455da96099c2b4766a59)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.248081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.312083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role cdffa045f237455da96099c2b4766a59 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:27.312754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:27.313352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:27.315700 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2740/5425] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:27 2022] DELETE /identity/v3/roles/cdffa045f237455da96099c2b4766a59 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:27.317422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.322410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.339535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d290b-b2d2-4dee-996b-beb9fd6fc0d3 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.346159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:27.402974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:27.420742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.421775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.423065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.423689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.426237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.446259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18400ed-ed8b-40cf-a747-1538075a1d8c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.452926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.453538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.453978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.454106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.454745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.457257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.470153 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2741/5426] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:27 2022] GET /identity/v3/roles => generated 1891 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:27.471491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.473272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.538904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.539825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.571862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2686/5427] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3578 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:27.574228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.578058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.635231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118d817-489a-4583-81bc-0e05592b615e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.640183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.640822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.641165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.641511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/regions/tempest-region-1478332577` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.642614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1478332577)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.643210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.649137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2742/5428] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:11:27 2022] GET /identity/v3/regions/tempest-region-1478332577 => generated 180 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:27.650039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.652518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:27.713676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bd23d-4cf9-4ad0-9069-7015fab48189 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.718721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.719857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.720429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.721194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/7f470358b2344d0abfb5ec4359a59660` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.731278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:get_role_for_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1, role_id=7f470358b2344d0abfb5ec4359a59660)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.733247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Could not find role: 7f470358b2344d0abfb5ec4359a59660.: keystone.exception.RoleNotFound: Could not find role: 7f470358b2344d0abfb5ec4359a59660. Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:11:27.747542 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 7f470358b2344d0abfb5ec4359a59660. Sep 24 21:11:27.748648 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:27.750541 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2687/5429] 10.222.0.52 () {58 vars in 1369 bytes} [Sat Sep 24 21:11:27 2022] HEAD /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1/roles/7f470358b2344d0abfb5ec4359a59660 => generated 0 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) Sep 24 21:11:27.751919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.770704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b30d5-0bda-41f3-9f57-5dfa99db7622 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.772279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d1eab-38ed-43de-a88f-fc858e9c45f9 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.781146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.783025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.783549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.785348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] PATH_INFO: `/v3/projects/b1ccbc91c8f0461b8a3de1250cbecfbd/users/68adaeee5bc9449cb6776d67eace2906/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.785731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.787347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.788405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.788685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:27.790746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.794820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.811658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ccbc91c8f0461b8a3de1250cbecfbd, user_id=68adaeee5bc9449cb6776d67eace2906, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:27.815298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:27.819560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2743/5430] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:27 2022] POST /identity/v3/roles => generated 239 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:27.819560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.832332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad268dcc-9383-4a60-8d20-b3e50d0ecdae tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:27.844253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2688/5431] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:27 2022] PUT /identity/v3/projects/b1ccbc91c8f0461b8a3de1250cbecfbd/users/68adaeee5bc9449cb6776d67eace2906/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:27.845608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:27.937067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:27.950437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:27.982111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:27.983542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:27.984197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:27.984840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints/a6628b829fa8455dbaa7f7dbc59eba87` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.005236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c0b6-6be9-446f-b381-38fc82b58f2f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.007655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=a6628b829fa8455dbaa7f7dbc59eba87)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.010449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.012603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.013070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.013070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.013070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.019351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorizing `identity:delete_trust(trust_id=89e3aba5ac604a76808613adb03eb0b1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: On instance['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: On instance['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: 'False' Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise error Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application 'False' Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.020748 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:11:28.022456 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application 'False' Sep 24 21:11:28.023583 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:28.024163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.031006 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2744/5432] 10.222.0.52 () {60 vars in 1255 bytes} [Sat Sep 24 21:11:27 2022] PATCH /identity/v3/endpoints/a6628b829fa8455dbaa7f7dbc59eba87 => generated 283 bytes in 214 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:11:28.031852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.060026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '89e3aba5ac604a76808613adb03eb0b1', 'request_id': 'req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.062955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '89e3aba5ac604a76808613adb03eb0b1', 'request_id': 'req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.083138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '89e3aba5ac604a76808613adb03eb0b1', 'request_id': 'req-de5396a5-063c-4ea7-8bf1-5f307e0e3b67'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.116950 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2689/5433] 10.222.0.52 () {58 vars in 1254 bytes} [Sat Sep 24 21:11:27 2022] DELETE /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.116950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.116950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.116950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.175708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8d41e-958f-4c7b-8163-80f1c2805728 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.192904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.193803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.194119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] MFA Rules not processed for user `68adaeee5bc9449cb6776d67eace2906`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:28.194255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.195412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.197731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.205940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f0b9854564924d0c9c1923e766e95764, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=f71adf53b99e462887551511943ba5fc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.209687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.234121 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2745/5434] 10.222.0.52 () {60 vars in 1582 bytes} [Sat Sep 24 21:11:28 2022] PUT /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.237765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.243021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.317447 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2690/5435] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3570 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:28.317447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.320601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.369340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad8d13-98d4-4937-8d72-08a7d2f137a8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.383220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efef4f3-49da-47ee-9dec-bc7b4188c86c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.390846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.392216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.392764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.394008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints/a6628b829fa8455dbaa7f7dbc59eba87` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.397512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=a6628b829fa8455dbaa7f7dbc59eba87)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.400078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.417406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'a6628b829fa8455dbaa7f7dbc59eba87', 'request_id': 'req-20f354ed-2e4f-4954-bbca-7f0944d73f8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.433445 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2746/5436] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:11:28 2022] DELETE /identity/v3/endpoints/a6628b829fa8455dbaa7f7dbc59eba87 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.435626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.437926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.471365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdaff246971c42d8bfb0526a4912b2bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fd4abcc7147859312b8eaaffa8cf9', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-816832148', 'tempest-DelegatedRole-534683317'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.472146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.472441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.472723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] PATH_INFO: `/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Could not find trust: 89e3aba5ac604a76808613adb03eb0b1.: keystone.exception.TrustNotFound: Could not find trust: 89e3aba5ac604a76808613adb03eb0b1. Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) Sep 24 21:11:28.487030 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 89e3aba5ac604a76808613adb03eb0b1. Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:28.488183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2691/5437] 10.222.0.52 () {58 vars in 1251 bytes} [Sat Sep 24 21:11:28 2022] GET /identity/v3/OS-TRUST/trusts/89e3aba5ac604a76808613adb03eb0b1 => generated 111 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:28.489889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.491085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.515222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:28.559734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f354ed-2e4f-4954-bbca-7f0944d73f8a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.568466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.569763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.570112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.570463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.572808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=f0b9854564924d0c9c1923e766e95764, group_id=ffb3cf6be1374d0596cf363525a6c24a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.574224 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:28.574224 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:28.574224 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:28.576297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.582848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083a042-7842-454d-92ab-36be80ba6903 tempest-TrustsV3TestJSON-1725188752 tempest-user-270570571] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.585181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:28.588421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.589033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.589314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.589633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.594997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2747/5438] 10.222.0.52 () {58 vars in 1464 bytes} [Sat Sep 24 21:11:28 2022] GET /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/inherited_to_projects => generated 447 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:28.600722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.629389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb954f-38de-4ff1-a701-950b5202d28d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.636240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.636450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.636789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.637065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/regions/tempest-region-1478332577` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.637702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1478332577)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.639484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.675873 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2748/5439] 10.222.0.52 () {58 vars in 1209 bytes} [Sat Sep 24 21:11:28 2022] DELETE /identity/v3/regions/tempest-region-1478332577 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.681558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.684479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.703173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.709203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.757600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2692/5440] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:28 2022] GET /identity/v3/auth/tokens => generated 3388 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:28.757600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.781948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82abb9f-684f-4dfd-af54-61b12708c7a2 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.803801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.804534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.804825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.805366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.809882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=f0b9854564924d0c9c1923e766e95764, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=f71adf53b99e462887551511943ba5fc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.818380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.826532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9e07f-c76d-4121-bda2-1170b5414caf tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.832086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.833269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.834223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.834749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/7f470358b2344d0abfb5ec4359a59660` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.835870 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2693/5441] 10.222.0.52 () {58 vars in 1564 bytes} [Sat Sep 24 21:11:28 2022] HEAD /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.837098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.837583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=7f470358b2344d0abfb5ec4359a59660)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.839469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.896429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc2d20-dc20-42b6-b120-a72276c89259 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:28.900679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role 7f470358b2344d0abfb5ec4359a59660 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:28.901776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.903055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:28.906806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2749/5442] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:28 2022] DELETE /identity/v3/roles/7f470358b2344d0abfb5ec4359a59660 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:28.908753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:28.909463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:28.909710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:28.909984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:28.910197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:28.910680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:28.912754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:28.917146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:28.961021 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-887060346. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:11:28.961021 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:11:28.961021 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:29.013753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2694/5443] 10.222.0.52 () {60 vars in 1156 bytes} [Sat Sep 24 21:11:28 2022] POST /identity/v3/endpoints => generated 358 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:29.015583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.019344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.034588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a1fe6-64c1-4af1-bddb-20b63891fc7a tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.063601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.064743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.065091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.065378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.070053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f0b9854564924d0c9c1923e766e95764, group_id=ffb3cf6be1374d0596cf363525a6c24a, role_id=f71adf53b99e462887551511943ba5fc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.074993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:29.077132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc7a2-f48b-462d-9835-6a1a41943355 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.082823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role f71adf53b99e462887551511943ba5fc was removed from group ffb3cf6be1374d0596cf363525a6c24a on domain f0b9854564924d0c9c1923e766e95764. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:29.083206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.087765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.115565 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2750/5444] 10.222.0.52 () {58 vars in 1566 bytes} [Sat Sep 24 21:11:28 2022] DELETE /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/groups/ffb3cf6be1374d0596cf363525a6c24a/roles/f71adf53b99e462887551511943ba5fc/inherited_to_projects => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:29.116584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.118120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.125493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:29.193749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:29.219597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.234200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.234490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.236922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/roles/766596adfe3e424ca85983fe765756a8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.242769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_role(role_id=766596adfe3e424ca85983fe765756a8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.244739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83dea6-fe1c-4e74-9c12-2fa53f65d6e7 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.248499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:29.257908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.260014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.260311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.260588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/regions/tempest-region-887060346` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.261965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-887060346)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.263075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:29.271182 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2751/5445] 10.222.0.52 () {58 vars in 1203 bytes} [Sat Sep 24 21:11:29 2022] GET /identity/v3/regions/tempest-region-887060346 => generated 178 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:29.273589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.276076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.352663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invalidating the token cache because role 766596adfe3e424ca85983fe765756a8 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:29.353203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.354050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.370632 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2695/5446] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:29 2022] DELETE /identity/v3/roles/766596adfe3e424ca85983fe765756a8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:29.372988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.382439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.463762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400276aa-5d57-4e23-925f-5dfa31c6db4c tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.592945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284a3ce-2e9e-49bb-b81c-e68d93bf3ce5 tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.631463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.631897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.632155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.632404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints/7eae97d1b6a24b038c254c229079902c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.643528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.646925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.648300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.648888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/f71adf53b99e462887551511943ba5fc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.651125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f71adf53b99e462887551511943ba5fc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.654080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:29.658894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=7eae97d1b6a24b038c254c229079902c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.660154 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:29.660154 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:29.660154 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:29.661070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: On instance['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: 'True' Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise error Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'True' Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.674964 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:11:29.677438 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['enabled']: Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'True' Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:11:29.679047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2696/5447] 10.222.0.52 () {60 vars in 1255 bytes} [Sat Sep 24 21:11:29 2022] PATCH /identity/v3/endpoints/7eae97d1b6a24b038c254c229079902c => generated 281 bytes in 308 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) Sep 24 21:11:29.680640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.681571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.752884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role f71adf53b99e462887551511943ba5fc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:29.753903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.758894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:29.758894 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2752/5448] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:29 2022] DELETE /identity/v3/roles/f71adf53b99e462887551511943ba5fc => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:29.758894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:29.763372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:29.785849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:29.886013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064ebe0-dcc0-4655-8e37-bd1b4c07b456 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.953016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a5c8-5f0a-4dc7-baa5-95f4a3737eab tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:29.960473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:29.961141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:29.961430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:29.961710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/endpoints/7eae97d1b6a24b038c254c229079902c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:29.963228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=7eae97d1b6a24b038c254c229079902c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:29.965894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:30.013298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '7eae97d1b6a24b038c254c229079902c', 'request_id': 'req-a94c4a37-d4c7-48f8-a420-93202da1481b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.015867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:30.029918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:30.030941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:30.031090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:30.031326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/users/bdaff246971c42d8bfb0526a4912b2bb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:30.035134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_user(user_id=bdaff246971c42d8bfb0526a4912b2bb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:30.040504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:30.046950 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2753/5449] 10.222.0.52 () {58 vars in 1236 bytes} [Sat Sep 24 21:11:29 2022] DELETE /identity/v3/endpoints/7eae97d1b6a24b038c254c229079902c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:30.046950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:30.056894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:30.221863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdaff246971c42d8bfb0526a4912b2bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.222548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdaff246971c42d8bfb0526a4912b2bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.249196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdaff246971c42d8bfb0526a4912b2bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.283660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdaff246971c42d8bfb0526a4912b2bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.286617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdaff246971c42d8bfb0526a4912b2bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.323669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4a37-d4c7-48f8-a420-93202da1481b tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:30.352687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:30.353324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:30.353604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:30.353875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:30.354631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:30.356608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:30.363172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2697/5450] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:29 2022] DELETE /identity/v3/users/bdaff246971c42d8bfb0526a4912b2bb => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:30.364165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:30.368243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:30.399301 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2754/5451] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:30 2022] POST /identity/v3/roles => generated 239 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:30.400339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:30.402765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:30.475303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adce8f-77ec-4de0-b42b-51cd4fb909ce tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:30.551432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735dc5d-2d74-4c52-ba9d-237a4895170c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:30.621868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:30.624351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:30.624728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:30.625281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/regions/tempest-region-887060346` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:30.625885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-887060346)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:30.627886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:30.675543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d9cb502b66409e803a84ae56a2754c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a77a8cca971441e1adc221f24d95799e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:30.678740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:30.679588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:30.680952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] PATH_INFO: `/v3/projects/064fd4abcc7147859312b8eaaffa8cf9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:30.685030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorizing `identity:delete_project(project_id=064fd4abcc7147859312b8eaaffa8cf9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:30.718059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2698/5452] 10.222.0.52 () {58 vars in 1206 bytes} [Sat Sep 24 21:11:30 2022] DELETE /identity/v3/regions/tempest-region-887060346 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:30.718669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:30.719861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:30.728020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:30.782173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:30.869443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064fd4abcc7147859312b8eaaffa8cf9', 'request_id': 'req-374e2bef-357e-4a79-8969-c10865fe369c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.875033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:30.894725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064fd4abcc7147859312b8eaaffa8cf9', 'request_id': 'req-374e2bef-357e-4a79-8969-c10865fe369c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.905388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064fd4abcc7147859312b8eaaffa8cf9', 'request_id': 'req-374e2bef-357e-4a79-8969-c10865fe369c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.932215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d90a4-1a49-4bb0-99d3-60e5381e9509 tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:30.950844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] The token cache is being invalidate because project 064fd4abcc7147859312b8eaaffa8cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:30.951304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.952102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:30.954043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2755/5453] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:30 2022] DELETE /identity/v3/projects/064fd4abcc7147859312b8eaaffa8cf9 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:30.954789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:30.960775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.030974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.032419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.032943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.033484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.036978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=4e3d2b1997364ed8a3573e32f28f5605)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.045358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.065257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2699/5454] 10.222.0.52 () {60 vars in 1579 bytes} [Sat Sep 24 21:11:30 2022] PUT /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:31.066658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:31.069054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.101124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2bef-357e-4a79-8969-c10865fe369c tempest-TrustsV3TestJSON-1696780939 tempest-TrustsV3TestJSON-1696780939-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:31.222571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce802418add84d59ab8a256ae1ea2e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13f013dc8b174609bc320320e3b034d0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.224433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.224950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.225522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] PATH_INFO: `/v3/services/fb3db3da4d0e4db5a4cc8d42653dc791` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.226571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorizing `identity:delete_service(service_id=fb3db3da4d0e4db5a4cc8d42653dc791)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.228632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.257105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e563932-061b-4dfc-8820-5343e459033f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:31.289401 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2756/5455] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:30 2022] DELETE /identity/v3/services/fb3db3da4d0e4db5a4cc8d42653dc791 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:31.289759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.289759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.290853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:31.291521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.291960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] PATH_INFO: `/v3/users/a8f280ab42484b9b95b64f1886088816` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.293146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.295859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f280ab42484b9b95b64f1886088816)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.305021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.396534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:31.444963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f280ab42484b9b95b64f1886088816'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.446425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f280ab42484b9b95b64f1886088816'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.465814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f280ab42484b9b95b64f1886088816'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.482623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f280ab42484b9b95b64f1886088816'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.483733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f280ab42484b9b95b64f1886088816'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.510654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:31.526635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc952c-021f-4eaf-bd7d-5b4338be600a tempest-EndpointsNegativeTestJSON-478982992 tempest-EndpointsNegativeTestJSON-478982992-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:31.532083 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2700/5456] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:31 2022] DELETE /identity/v3/users/a8f280ab42484b9b95b64f1886088816 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:31.533956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:31.536086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.677328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.678128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.678469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.678854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.683909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.687828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a737d-01ee-46b3-9084-5045e172b996 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.696525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f66f3-2a56-47f5-87ed-c6afcfda2503 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:31.724572 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2757/5457] 10.222.0.52 () {58 vars in 1461 bytes} [Sat Sep 24 21:11:31 2022] GET /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/inherited_to_projects => generated 446 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:31.724572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.724572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.724572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.732115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.740389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.740757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.741164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] PATH_INFO: `/v3/users/3af31013bd244b14ac9bf47b7d9fe392` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.743534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af31013bd244b14ac9bf47b7d9fe392)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.745566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.777438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:31.779789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.813777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af31013bd244b14ac9bf47b7d9fe392'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.814950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af31013bd244b14ac9bf47b7d9fe392'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.817212 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2758/5458] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:31.818871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:31.820661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.834661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af31013bd244b14ac9bf47b7d9fe392'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.849774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af31013bd244b14ac9bf47b7d9fe392'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.851513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af31013bd244b14ac9bf47b7d9fe392'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:31.856869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c7f3-8d5b-4805-a5a4-cec20f3929b3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:31.878861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:31.879754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.881275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.882813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.887969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=4e3d2b1997364ed8a3573e32f28f5605)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:31.894769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afbc30-8905-43fd-a1fa-ab6d7bd67a5e tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:31.901271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2701/5459] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:31 2022] DELETE /identity/v3/users/3af31013bd244b14ac9bf47b7d9fe392 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:31.903849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:31.908205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:31.917677 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2759/5460] 10.222.0.52 () {58 vars in 1561 bytes} [Sat Sep 24 21:11:31 2022] HEAD /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:31.917677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:31.917677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:31.917677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:31.955571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:31.957639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.063573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696997-f5b4-4783-9655-d836b40a2705 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.073437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.074064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.074340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.074944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.080584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.104092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:32.127099 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2760/5461] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:32.129778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.134955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.220748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.223015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.228176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:32.247197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2702/5462] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:32.248147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.260704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c83a3b-4c02-413a-b1ad-a73dd871041a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.271038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.272637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.273034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.273475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.273605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd7a5-3246-4495-bed4-ef899620f6b7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.276049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=4e3d2b1997364ed8a3573e32f28f5605)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.279244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.283758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.285591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 4e3d2b1997364ed8a3573e32f28f5605 was removed from user a592b7077dc3419ea466c6d39fbc3fc0 on domain f0b9854564924d0c9c1923e766e95764. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:32.285992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:32.287271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.287570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:32.287707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.287979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.293092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.314833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.315432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2761/5463] 10.222.0.52 () {58 vars in 1563 bytes} [Sat Sep 24 21:11:32 2022] DELETE /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/4e3d2b1997364ed8a3573e32f28f5605/inherited_to_projects => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:32.316896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.319644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.321681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.333646 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2703/5464] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:32.438000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfbfcd-9cd7-49bc-87f7-c9d3ffcc86bc tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.445454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.446351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.446896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.447585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/4e3d2b1997364ed8a3573e32f28f5605` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.449456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4e3d2b1997364ed8a3573e32f28f5605)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.451877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.550012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 4e3d2b1997364ed8a3573e32f28f5605 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:32.551029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:32.554539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:32.563635 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2762/5465] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:32 2022] DELETE /identity/v3/roles/4e3d2b1997364ed8a3573e32f28f5605 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:32.565939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.591626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd351a0-2103-49c5-b648-7738c3fbf466 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.740868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.741848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.741848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.742317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.744121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.747890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.782140 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2704/5466] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:32 2022] POST /identity/v3/roles => generated 240 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:32.789863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.792895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:32.812147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a405230-9907-47a0-b9fc-3e389a2ce3c4 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:32.824934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:32.826020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:32.827838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:32.827838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:32.828569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:32.832141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:32.916627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2763/5467] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:32 2022] POST /identity/v3/projects => generated 407 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:32.920281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:32.938389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb1bb3-b77b-437a-9fb7-10f2e2839261 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.087187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.087983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.088218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.088642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/6e9d447d7d084de7a584b1484c9f7393/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.094596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=6e9d447d7d084de7a584b1484c9f7393)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.101488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:33.136017 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2705/5468] 10.222.0.52 () {60 vars in 1579 bytes} [Sat Sep 24 21:11:32 2022] PUT /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/6e9d447d7d084de7a584b1484c9f7393/inherited_to_projects => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:33.137432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:33.159510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740df95b-4be6-4892-8a19-3ce8357dd493 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.307949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.308588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.308868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.309152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.310266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=5376a1c6d7e145b1aca1cf1b936d0acf, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:33.310266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.311597 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:33.311597 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:33.311597 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:33.312815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:33.330930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:33.399422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2764/5469] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:33 2022] GET /identity/v3/role_assignments?scope.project.id=5376a1c6d7e145b1aca1cf1b936d0acf&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 648 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:33.404129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:33.420848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871ee8b-efaa-4321-b513-e757ef2a2cec tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.436129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.437021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.437147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.437318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.438209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=a71f89a0ab544b4d919eaf34c5dad0d4, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:33.438358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.440924 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:33.440924 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:33.440924 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:33.444988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:33.499932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:33.581015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:33.585996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:33.628590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2706/5470] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:33 2022] GET /identity/v3/role_assignments?scope.project.id=a71f89a0ab544b4d919eaf34c5dad0d4&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 648 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:33.633912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:33.657623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3659b-14fc-45fd-9103-77b35163197c tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.673363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.674515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.676318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.676883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/6e9d447d7d084de7a584b1484c9f7393/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.685573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=f0b9854564924d0c9c1923e766e95764, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=6e9d447d7d084de7a584b1484c9f7393)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.691701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:33.708551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 6e9d447d7d084de7a584b1484c9f7393 was removed from user a592b7077dc3419ea466c6d39fbc3fc0 on domain f0b9854564924d0c9c1923e766e95764. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:33.708945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:33.713583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:33.753571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2707/5471] 10.222.0.52 () {58 vars in 1563 bytes} [Sat Sep 24 21:11:33 2022] DELETE /identity/v3/OS-INHERIT/domains/f0b9854564924d0c9c1923e766e95764/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/6e9d447d7d084de7a584b1484c9f7393/inherited_to_projects => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:33.753805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fb40-ff05-4f51-a126-194c68724cc0 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.755813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:33.758808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:33.915702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5999-d6af-4ff9-a99e-0f0ff9291b99 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:33.915969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.917393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.918851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.919137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] PATH_INFO: `/v3/projects/8d0cb5ff1eb0440abace7fc7d9bb16ef` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.922327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:33.923020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:33.923163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0cb5ff1eb0440abace7fc7d9bb16ef)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.923390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:33.923671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:33.924618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=5376a1c6d7e145b1aca1cf1b936d0acf, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:33.925209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:33.927645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:33.929600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:34.027912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0cb5ff1eb0440abace7fc7d9bb16ef', 'request_id': 'req-3924ab89-8d9a-4458-a720-3c05a97a2fdf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.042633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0cb5ff1eb0440abace7fc7d9bb16ef', 'request_id': 'req-3924ab89-8d9a-4458-a720-3c05a97a2fdf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.056465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0cb5ff1eb0440abace7fc7d9bb16ef', 'request_id': 'req-3924ab89-8d9a-4458-a720-3c05a97a2fdf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.069258 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2708/5472] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:33 2022] GET /identity/v3/role_assignments?scope.project.id=5376a1c6d7e145b1aca1cf1b936d0acf&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 228 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:34.073165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.079131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] The token cache is being invalidate because project 8d0cb5ff1eb0440abace7fc7d9bb16ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:34.079680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.080620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:34.081058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.085130 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2765/5473] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:33 2022] DELETE /identity/v3/projects/8d0cb5ff1eb0440abace7fc7d9bb16ef => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:34.086194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.088705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:34.100006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:34.105743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe831b46-2457-4a7e-ba26-96c5cc0b012d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:34.200828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:34.202145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:34.202858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:34.203786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] PATH_INFO: `/v3/projects/5fa863cc66874d03889b96d25a37e606` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:34.205459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa863cc66874d03889b96d25a37e606)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:34.205704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924ab89-8d9a-4458-a720-3c05a97a2fdf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:34.210428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:34.212584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:34.213963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:34.214202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:34.214614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:34.215405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=a71f89a0ab544b4d919eaf34c5dad0d4, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:34.215715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:34.219270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:34.233270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:34.327584 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2766/5474] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:34 2022] GET /identity/v3/role_assignments?scope.project.id=a71f89a0ab544b4d919eaf34c5dad0d4&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 228 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:34.327584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.329855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:34.336165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa863cc66874d03889b96d25a37e606', 'request_id': 'req-d984b2db-4fd4-4638-b0fc-75ca8611f837'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.362299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa863cc66874d03889b96d25a37e606', 'request_id': 'req-d984b2db-4fd4-4638-b0fc-75ca8611f837'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.380923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436d1d7-9abc-4292-9f87-9794b38cb99f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:34.383787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa863cc66874d03889b96d25a37e606', 'request_id': 'req-d984b2db-4fd4-4638-b0fc-75ca8611f837'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.438800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] The token cache is being invalidate because project 5fa863cc66874d03889b96d25a37e606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:34.440253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.442075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.459445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2709/5475] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:34 2022] DELETE /identity/v3/projects/5fa863cc66874d03889b96d25a37e606 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:34.461819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.486798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:34.498896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:34.501057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:34.501948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:34.502884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] PATH_INFO: `/v3/users/f6d9cb502b66409e803a84ae56a2754c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:34.507056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d9cb502b66409e803a84ae56a2754c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:34.511710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:34.624020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d9cb502b66409e803a84ae56a2754c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.625525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d9cb502b66409e803a84ae56a2754c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.663447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d9cb502b66409e803a84ae56a2754c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.691668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b2db-4fd4-4638-b0fc-75ca8611f837 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:34.691935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:34.703314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:34.704199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:34.704468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:34.704683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects/a71f89a0ab544b4d919eaf34c5dad0d4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:34.709325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a71f89a0ab544b4d919eaf34c5dad0d4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:34.713289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:34.717038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d9cb502b66409e803a84ae56a2754c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.718340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d9cb502b66409e803a84ae56a2754c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.752845 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2767/5476] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:34 2022] DELETE /identity/v3/users/f6d9cb502b66409e803a84ae56a2754c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:34.754048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.758708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:34.821103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71f89a0ab544b4d919eaf34c5dad0d4', 'request_id': 'req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.841884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71f89a0ab544b4d919eaf34c5dad0d4', 'request_id': 'req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.865842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71f89a0ab544b4d919eaf34c5dad0d4', 'request_id': 'req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.932831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] The token cache is being invalidate because project a71f89a0ab544b4d919eaf34c5dad0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:34.937803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.943506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:34.955798 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2710/5477] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:34 2022] DELETE /identity/v3/projects/a71f89a0ab544b4d919eaf34c5dad0d4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:34.975002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:34.985200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:35.038995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2341ea-bafe-4745-939a-bd9ed4599e67 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:35.099233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:35.185792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6ba05-8bc0-43e2-b4de-1df96e332f14 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:35.202177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:35.203178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:35.203611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:35.203899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:35.204250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] PATH_INFO: `/v3/users/ce802418add84d59ab8a256ae1ea2e1e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:35.204689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:35.205189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:35.205472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/6e9d447d7d084de7a584b1484c9f7393` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:35.205899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce802418add84d59ab8a256ae1ea2e1e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:35.208649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6e9d447d7d084de7a584b1484c9f7393)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:35.209259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:35.212778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:35.301362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 6e9d447d7d084de7a584b1484c9f7393 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:35.301851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.308061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.318022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2711/5478] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:34 2022] DELETE /identity/v3/roles/6e9d447d7d084de7a584b1484c9f7393 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:35.322129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:35.331854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:35.347645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce802418add84d59ab8a256ae1ea2e1e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.349127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce802418add84d59ab8a256ae1ea2e1e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.368365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce802418add84d59ab8a256ae1ea2e1e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.426568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce802418add84d59ab8a256ae1ea2e1e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.433809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce802418add84d59ab8a256ae1ea2e1e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:35.462330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2768/5479] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:34 2022] DELETE /identity/v3/users/ce802418add84d59ab8a256ae1ea2e1e => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:35.555247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718168b7-bac3-45fc-9e04-50cf87cba89a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:35.711151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:35.711778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:35.712056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:35.712335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:35.713064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:35.714875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:35.744177 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2712/5480] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:35 2022] POST /identity/v3/roles => generated 240 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:35.749851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:35.751774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:35.773562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:35.775845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20b6c1-2ea1-46be-941e-e21f19f6f1e5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:35.782737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:35.783525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:35.783916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:35.784437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:35.785552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:35.794149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:35.894786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2769/5481] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:35 2022] POST /identity/v3/projects => generated 409 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:35.898785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:35.912570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9a797-f920-403f-89ce-a43a6fabba80 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:35.991616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:36.048831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:36.049528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.049815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.050099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/040aa4dcd3b1481681a9d997d34e850a/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.053504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:create_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=040aa4dcd3b1481681a9d997d34e850a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:36.070406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:36.071597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:36.073963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:36.093235 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2713/5482] 10.222.0.52 () {60 vars in 1582 bytes} [Sat Sep 24 21:11:35 2022] PUT /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/040aa4dcd3b1481681a9d997d34e850a/inherited_to_projects => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:36.103545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:36.230607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d674-1c50-4a2e-b94f-ad1b07e3c1c5 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:36.273708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4ecb-8b28-4447-b914-6cf85ea446e9 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:36.281612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:36.282247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.283077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.283077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.283995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=9af80ccf0c8c46d5807199bb498548fd, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:36.293285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:36.295998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:36.352810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:36.353464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.353987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.354656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] PATH_INFO: `/v3/projects/a77a8cca971441e1adc221f24d95799e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.356171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77a8cca971441e1adc221f24d95799e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:36.374624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:36.454209 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2714/5483] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:36 2022] GET /identity/v3/role_assignments?scope.project.id=9af80ccf0c8c46d5807199bb498548fd&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 649 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:36.456363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:36.475896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44d12-5164-49da-b8d8-8bf28c525e9a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:36.501298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77a8cca971441e1adc221f24d95799e', 'request_id': 'req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.533276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77a8cca971441e1adc221f24d95799e', 'request_id': 'req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.547551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77a8cca971441e1adc221f24d95799e', 'request_id': 'req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.581721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] The token cache is being invalidate because project a77a8cca971441e1adc221f24d95799e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:36.582620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.584543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea0a25b8-6845-44b4-bd76-2c23704c1c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.588789 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2770/5484] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:36 2022] DELETE /identity/v3/projects/a77a8cca971441e1adc221f24d95799e => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:36.593602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc303ce-2ca2-42b7-898e-d89b37d2aaa5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.593971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc303ce-2ca2-42b7-898e-d89b37d2aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.594331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc303ce-2ca2-42b7-898e-d89b37d2aaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.623423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:36.624035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.624321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.624605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/040aa4dcd3b1481681a9d997d34e850a/inherited_to_projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.629608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=5376a1c6d7e145b1aca1cf1b936d0acf, user_id=a592b7077dc3419ea466c6d39fbc3fc0, role_id=040aa4dcd3b1481681a9d997d34e850a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:36.636735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:36.637727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0fc303ce-2ca2-42b7-898e-d89b37d2aaa5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:36.639069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0fc303ce-2ca2-42b7-898e-d89b37d2aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:36.650925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 040aa4dcd3b1481681a9d997d34e850a was removed from user a592b7077dc3419ea466c6d39fbc3fc0 on project 5376a1c6d7e145b1aca1cf1b936d0acf. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:36.651253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.652741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:36.653386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:36.686645 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2715/5485] 10.222.0.52 () {58 vars in 1566 bytes} [Sat Sep 24 21:11:36 2022] DELETE /identity/v3/OS-INHERIT/projects/5376a1c6d7e145b1aca1cf1b936d0acf/users/a592b7077dc3419ea466c6d39fbc3fc0/roles/040aa4dcd3b1481681a9d997d34e850a/inherited_to_projects => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:36.691300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:36.694616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:36.736175 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2771/5486] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:36.745738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.745951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.746324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.746460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:36.782407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:36.783622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:36.866239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578b23-e86d-4e0a-8c99-041698462045 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:36.874421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:36.875773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:36.876377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:36.876982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:36.878276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Adding query filter params (scope.project.id=9af80ccf0c8c46d5807199bb498548fd, user.id=a592b7077dc3419ea466c6d39fbc3fc0) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:36.878873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:36.882026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:36.959955 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2772/5487] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:36.963205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:36.963901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:36.993183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6469f6-e981-4745-8dfb-cbaf15eb5bc9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.001022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2716/5488] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:11:36 2022] GET /identity/v3/role_assignments?scope.project.id=9af80ccf0c8c46d5807199bb498548fd&user.id=a592b7077dc3419ea466c6d39fbc3fc0&effective => generated 228 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:37.002521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.007934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.008895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.009268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.009786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] PATH_INFO: `/v3/projects/13f013dc8b174609bc320320e3b034d0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.012833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f013dc8b174609bc320320e3b034d0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.017037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.044329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473df778-3935-4f2b-b0a7-b6cd4c940655 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.056894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.057668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.058060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.058557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.062942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:37.064012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.066331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.085777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2717/5489] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:37.086876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.104023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871b22-55fb-4ecd-964d-0968c7f68a22 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.124553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f013dc8b174609bc320320e3b034d0', 'request_id': 'req-fd4b4d0e-98e1-49b4-b322-0d916da397a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.138310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f013dc8b174609bc320320e3b034d0', 'request_id': 'req-fd4b4d0e-98e1-49b4-b322-0d916da397a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.152725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f013dc8b174609bc320320e3b034d0', 'request_id': 'req-fd4b4d0e-98e1-49b4-b322-0d916da397a0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.192816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] The token cache is being invalidate because project 13f013dc8b174609bc320320e3b034d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:37.193295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.194291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.199002 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2773/5490] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:36 2022] DELETE /identity/v3/projects/13f013dc8b174609bc320320e3b034d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:37.200008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.201387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.217499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.219086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.219517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.219952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects/9af80ccf0c8c46d5807199bb498548fd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.222104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9af80ccf0c8c46d5807199bb498548fd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.226190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.277401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:37.294750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9af80ccf0c8c46d5807199bb498548fd', 'request_id': 'req-1faba4f9-395b-421d-82f3-ad774e4957ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.307201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9af80ccf0c8c46d5807199bb498548fd', 'request_id': 'req-1faba4f9-395b-421d-82f3-ad774e4957ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.312874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:37.329433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9af80ccf0c8c46d5807199bb498548fd', 'request_id': 'req-1faba4f9-395b-421d-82f3-ad774e4957ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.344107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4d0e-98e1-49b4-b322-0d916da397a0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.346940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] The token cache is being invalidate because project 9af80ccf0c8c46d5807199bb498548fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:37.347246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.362612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1faba4f9-395b-421d-82f3-ad774e4957ed tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.362612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2718/5491] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:37 2022] DELETE /identity/v3/projects/9af80ccf0c8c46d5807199bb498548fd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:37.362612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.362612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.362612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.390728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:37.392506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.462563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.463246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.463528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.463973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.465233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.467385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc794a9-4777-4464-add2-6a6d3060b6a9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.506976 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2719/5492] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:37.508142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.511680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.525134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2774/5493] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:37 2022] POST /identity/v3/projects => generated 367 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:37.526238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.526544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.526818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.585655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:37.587607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.620045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082afc19-922c-497b-8a1b-4241cd44c01b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.732799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.733631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.733919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.734201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/roles/040aa4dcd3b1481681a9d997d34e850a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.738943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_role(role_id=040aa4dcd3b1481681a9d997d34e850a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.738943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.739392 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2775/5494] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:37.739911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.745779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.775759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455b656-fda3-46b3-8b46-47b9e5e0f7dc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.784082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.784766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.785003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.785209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.786213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.788832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.811354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invalidating the token cache because role 040aa4dcd3b1481681a9d997d34e850a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:37.811742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.812469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:37.816683 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2720/5495] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:37 2022] DELETE /identity/v3/roles/040aa4dcd3b1481681a9d997d34e850a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:37.818792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.825345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.826157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2776/5496] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:37 2022] POST /identity/v3/users => generated 284 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:37.828795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.832333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:37.838597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:37.871520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a0ff2-2f7b-4bdd-a343-361903d1da36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.960458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b0064-84e2-41f0-92c2-fb1cdf39eabf tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:37.972726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:37.973410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:37.973983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:37.974266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:37.975126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:37.975440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:37.978603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:37.984802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:37.997848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2721/5497] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:37.999289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:37.999427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.000012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.001702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.001996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/users/a592b7077dc3419ea466c6d39fbc3fc0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.003329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.011359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a592b7077dc3419ea466c6d39fbc3fc0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.013321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.020894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64113-cb07-4cda-b28c-fb27db3ce7e4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.027215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.028070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.028661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.029140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.030040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.032356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.047291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2722/5498] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:38.048232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.063624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95e980-bdc7-428b-82df-2ed73bfe671b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.070772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.071416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.071885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.072320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.072967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.075533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.095386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a592b7077dc3419ea466c6d39fbc3fc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:38.096390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a592b7077dc3419ea466c6d39fbc3fc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:38.118547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a592b7077dc3419ea466c6d39fbc3fc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:38.165639 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2723/5499] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:38 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:38.165639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.169676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.176334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a592b7077dc3419ea466c6d39fbc3fc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:38.178422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a592b7077dc3419ea466c6d39fbc3fc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:38.204967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2777/5500] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:37 2022] DELETE /identity/v3/users/a592b7077dc3419ea466c6d39fbc3fc0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:38.205978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.209267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.290020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63809-c299-4b4b-bd9e-5eb01876c25b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.298989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5b26d-ec06-4199-bb91-e60f9c75f6b8 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.309826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.310461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.312186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.312476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.313157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.314803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.334349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2778/5501] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:38.335266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.337073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.356504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fdec5-7c94-406e-9c7e-f3a13b6dc750 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.362817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.363443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.364573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.364851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.365600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.367748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.404174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.404795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.405040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.405752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.407755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.421950 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2779/5502] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:38 2022] POST /identity/v3/users => generated 285 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:38.422928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.429100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.430647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.430805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.443675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2724/5503] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:38.444538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.459002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2884e8-8557-4d77-bc3c-ea87f2780c18 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.466384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.467318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.467659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.467999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] PATH_INFO: `/v3/projects/21cf53366d5e4236800a13b32efcf11b/users/395c85cb5dff4c31a371d2d39fe594a2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.488176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cf53366d5e4236800a13b32efcf11b, user_id=395c85cb5dff4c31a371d2d39fe594a2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.492360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.505618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:38.513599 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2725/5504] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:38 2022] PUT /identity/v3/projects/21cf53366d5e4236800a13b32efcf11b/users/395c85cb5dff4c31a371d2d39fe594a2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:38.524593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.621724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:38.650468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f42b5-7dfd-406d-ab0d-bd0b561d017a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.652700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c7a0-00cb-4323-a971-fd9221c71739 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.661874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.663027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.663593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.664012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.665387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.668128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c5ed-7b0f-4469-bc17-eda103b01054 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.691800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2726/5505] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:38.691800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.691800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.691800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.745202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] MFA Rules not processed for user `395c85cb5dff4c31a371d2d39fe594a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:38.747619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:38.776775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.777359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.777645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.777923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/groups/ffb3cf6be1374d0596cf363525a6c24a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.779190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_group(group_id=ffb3cf6be1374d0596cf363525a6c24a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.780476 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:38.780476 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:38.780476 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:38.782609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.895190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2727/5506] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:38.896946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.913871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28c78-0366-4a8a-a267-223e0869e621 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:38.921814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2780/5507] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:11:38 2022] DELETE /identity/v3/groups/ffb3cf6be1374d0596cf363525a6c24a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:38.927171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.939370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:38.940006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:38.940290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:38.940562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:38.941217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:38.943500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:38.954890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2728/5508] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:38.955870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:38.993411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab5088-f109-42b5-94a2-ff4333df2b9a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.069115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a05d4b-34cd-4be2-a6aa-a13c7f919fd3 tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.078193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.079124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.079758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.080640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.081849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.085388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.115857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.116505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.116849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.117126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/projects/5376a1c6d7e145b1aca1cf1b936d0acf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.118366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5376a1c6d7e145b1aca1cf1b936d0acf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.122561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.139821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2781/5509] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:38 2022] POST /identity/v3/projects => generated 365 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:39.140673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.153854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:39.215613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5376a1c6d7e145b1aca1cf1b936d0acf', 'request_id': 'req-40163f72-a15a-406f-a14c-6c0fea8f829d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.231098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5376a1c6d7e145b1aca1cf1b936d0acf', 'request_id': 'req-40163f72-a15a-406f-a14c-6c0fea8f829d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.248405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:39.257026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5376a1c6d7e145b1aca1cf1b936d0acf', 'request_id': 'req-40163f72-a15a-406f-a14c-6c0fea8f829d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.273429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] The token cache is being invalidate because project 5376a1c6d7e145b1aca1cf1b936d0acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:39.273725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.274696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.277129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca0a2a-3280-486b-8370-8761f2884c25 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.280695 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2729/5510] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:38 2022] DELETE /identity/v3/projects/5376a1c6d7e145b1aca1cf1b936d0acf => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:39.281872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.283618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.361929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.362834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.363120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.363394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] PATH_INFO: `/v3/projects/c78df8495b6849d58b71a6ccbf5041ee/users/f1e98565112a4b20b910b1bc8fa1a7e3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.389561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78df8495b6849d58b71a6ccbf5041ee, user_id=f1e98565112a4b20b910b1bc8fa1a7e3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.392910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.402997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40163f72-a15a-406f-a14c-6c0fea8f829d tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.408592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.409333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.409666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.409947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.410637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.416854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f8660-c188-4492-942f-cb8177eb622f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.422236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2782/5511] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:39 2022] PUT /identity/v3/projects/c78df8495b6849d58b71a6ccbf5041ee/users/f1e98565112a4b20b910b1bc8fa1a7e3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:39.423221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.433971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.466360 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2730/5512] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:39 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:39.468914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.470061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.470326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.529509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6740-b2f7-4054-84cf-4bb908e39d72 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.535712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.537198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.537480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.537790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/domains/f0b9854564924d0c9c1923e766e95764` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.539139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f0b9854564924d0c9c1923e766e95764)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.539269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] MFA Rules not processed for user `f1e98565112a4b20b910b1bc8fa1a7e3`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:39.540617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.542071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.574300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.575245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.618251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.619839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:39.630325 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2783/5513] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:11:39 2022] PATCH /identity/v3/domains/f0b9854564924d0c9c1923e766e95764 => generated 276 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:39.632273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.633332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.706370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:39.709378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2731/5514] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:39.729359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.729359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.765500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd4f1e-4577-4c94-9335-3dce637aff7a tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.769636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2a9ff-63df-4ee0-b84d-3d722eb09e93 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.776525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.777171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.777452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.777725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.778366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.780306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.812445 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2784/5515] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:39.813315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.815293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:39.823624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:39.833795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca09fb-ba36-4305-bf12-95d29faa2cc7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.840533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.841505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.841996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.842375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.843191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.845323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.889873 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2785/5516] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:39 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:39.893490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:39.916427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ada02cf79d6453b90cd6ed4686dced7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f39f22cf5bb04332a9c8579ff38d859c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.917774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.918487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.920179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] PATH_INFO: `/v3/domains/f0b9854564924d0c9c1923e766e95764` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.963989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f0b9854564924d0c9c1923e766e95764)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:39.967122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:39.975173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b96b8-34ab-43b9-a57b-ed2308f8796d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:39.981908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:39.982803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:39.983153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:39.983499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] PATH_INFO: `/v3/projects/ff4a6b8f1b274032955ba99fee794874/users/5a09c6192d0a42beabe0f847fef930a7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:39.999593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.002966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4a6b8f1b274032955ba99fee794874, user_id=5a09c6192d0a42beabe0f847fef930a7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.005878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.016022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f0b9854564924d0c9c1923e766e95764'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.037515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2786/5517] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:39 2022] PUT /identity/v3/projects/ff4a6b8f1b274032955ba99fee794874/users/5a09c6192d0a42beabe0f847fef930a7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:40.042484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.104120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.113187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.139497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.156972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59efe3d-acef-4b58-8bd7-9c5cb51fc35f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.179147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] The token cache is being invalidate because project f0b9854564924d0c9c1923e766e95764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:40.179687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.181279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.203927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.205498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.213722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f0b9854564924d0c9c1923e766e95764', 'request_id': 'req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.226710 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2732/5518] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:11:39 2022] DELETE /identity/v3/domains/f0b9854564924d0c9c1923e766e95764 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:40.227977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.229993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.258578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:40.297600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.298210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.298513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.299000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.299419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.306289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.361284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2787/5519] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:40 2022] POST /identity/v3/users => generated 291 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:40.362346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.367464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.392377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ad6cb-66c1-4c94-886d-39fde9cc6e43 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.396656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4dd4a-4f74-423b-9c9f-064f389e6e7f tempest-InheritsV3TestJSON-150603878 tempest-InheritsV3TestJSON-150603878-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.401949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.402680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.403109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.403379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.404100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.406006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.407549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.407721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:40.408641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.410625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.411467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] PATH_INFO: `/v3/users/c165554c58674e57a655606a401a0f3b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.414357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c165554c58674e57a655606a401a0f3b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.419482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.420946 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2733/5520] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:40.428112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.431266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.449312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607aa2-7258-4ad7-ac6b-275555c97229 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.455776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.456675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.457025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.457373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.458166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.460073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.474554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2734/5521] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:40.475000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.490850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eabbca-eb4c-4a99-86fe-043732e8d2b0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.497654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.498705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.499072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.499463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] PATH_INFO: `/v3/projects/ff4a6b8f1b274032955ba99fee794874/users/5a09c6192d0a42beabe0f847fef930a7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.501890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4a6b8f1b274032955ba99fee794874, user_id=5a09c6192d0a42beabe0f847fef930a7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.506143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.524828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c165554c58674e57a655606a401a0f3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.525819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c165554c58674e57a655606a401a0f3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.539477 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2735/5522] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:40 2022] PUT /identity/v3/projects/ff4a6b8f1b274032955ba99fee794874/users/5a09c6192d0a42beabe0f847fef930a7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:40.540553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.540951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c165554c58674e57a655606a401a0f3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.543458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.599262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c165554c58674e57a655606a401a0f3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.603520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fe7be81b-4468-4144-ab87-7834a8ade582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c165554c58674e57a655606a401a0f3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:40.627033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2788/5523] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:40 2022] DELETE /identity/v3/users/c165554c58674e57a655606a401a0f3b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:40.628341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363dec-d924-49e7-a2fb-d1c6355116aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.628626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363dec-d924-49e7-a2fb-d1c6355116aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.628899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363dec-d924-49e7-a2fb-d1c6355116aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.682115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ef0e6-60a4-4379-b4f9-76d5dcddd909 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:40.695658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9c363dec-d924-49e7-a2fb-d1c6355116aa None None] MFA Rules not processed for user `5a09c6192d0a42beabe0f847fef930a7`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:40.696935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c363dec-d924-49e7-a2fb-d1c6355116aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.841483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:40.842169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.842444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.842774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] PATH_INFO: `/v3/projects/bfd69ea77d724d298f60f66b410c1b93/users/d97d0bc50fed4f15af1029f6b52740f2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.855025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2789/5524] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:40.856345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:40.856651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:40.856912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:40.879018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd69ea77d724d298f60f66b410c1b93, user_id=d97d0bc50fed4f15af1029f6b52740f2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:40.883677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:40.895654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:40.896976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:40.912314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2736/5525] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:40 2022] PUT /identity/v3/projects/bfd69ea77d724d298f60f66b410c1b93/users/d97d0bc50fed4f15af1029f6b52740f2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:40.913736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:40.938488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:40.991390 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2790/5526] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:40.992410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.017677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b43bc3-4812-4931-9923-bb98e34f4824 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.032330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.033043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.033458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.033976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.037092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.040024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.055734 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2791/5527] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:41.055734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.065440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea965a6-f785-44c6-99e7-ec86d20546da admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.068277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:41.077353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.078856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.079639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.079917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.082078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.085794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.096366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:41.167629 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2737/5528] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:40 2022] POST /identity/v3/projects => generated 359 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:41.169634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.199845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22bf69-af34-4759-b136-62c837972fab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.255365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc44ba6-92a2-49fe-957c-99abb8fa0f54 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.260785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.262623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.263019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.263524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] PATH_INFO: `/v3/projects/bfd69ea77d724d298f60f66b410c1b93/users/d97d0bc50fed4f15af1029f6b52740f2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.265753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd69ea77d724d298f60f66b410c1b93, user_id=d97d0bc50fed4f15af1029f6b52740f2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.269590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.272153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.272843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.273195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.273536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.285183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2738/5529] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:41 2022] PUT /identity/v3/projects/bfd69ea77d724d298f60f66b410c1b93/users/d97d0bc50fed4f15af1029f6b52740f2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:41.286702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.286918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.288624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba065db3-9f55-4fb4-be8b-1ae9f924c8d8 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.299502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2792/5530] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:41.300844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.301194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.301578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.331863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] MFA Rules not processed for user `d97d0bc50fed4f15af1029f6b52740f2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:41.333784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:41.399509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90012f2c-59fc-47b8-9b29-c2a671f795cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.413541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.413856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.414657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.414657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.415098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.419098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.429686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2793/5531] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:41.435079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.447431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:41.461126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c18c2a-20e3-4cbc-ba3e-c258c3479e04 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.473070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.474237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.478149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.479172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.482197 ncdv-master-1 devstack@keystone.service[99132]: WARNING oslo_log.versionutils [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 24 21:11:41.482745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.485036 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:41.485036 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:41.485036 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:41.486068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.492681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2739/5532] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:41 2022] POST /identity/v3/users => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:41.496205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.509306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3435-d3e8-4bb5-b92b-66ea9c1780f2 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.514751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.515413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.516795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.516795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/5007fd2fd5c745e4b08af70b4ebf162a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.534711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_user(user_id=5007fd2fd5c745e4b08af70b4ebf162a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.538183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.540703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2740/5533] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:11:41 2022] GET /identity/v3/users/5007fd2fd5c745e4b08af70b4ebf162a => generated 373 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:11:41.549190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.550184 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2794/5534] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:41 2022] POST /identity/v3/policies => generated 229 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:11:41.553206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.568651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6d185-097f-48c5-829b-478dddc0d094 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.571469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f65946-1068-46a8-8023-c715e983b7d1 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.573880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.574708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.575072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.575406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/5007fd2fd5c745e4b08af70b4ebf162a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.576644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_user(user_id=5007fd2fd5c745e4b08af70b4ebf162a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.578777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.579052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.579939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.580728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.581211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/4663046a4655413fb59199e7c254bdd4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.583875 ncdv-master-1 devstack@keystone.service[99132]: WARNING oslo_log.versionutils [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 24 21:11:41.586305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=4663046a4655413fb59199e7c254bdd4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.588559 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:41.588559 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:41.588559 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:41.589683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.629152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2795/5535] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:11:41 2022] PATCH /identity/v3/policies/4663046a4655413fb59199e7c254bdd4 => generated 236 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:41.635290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.667992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544dbc1-a703-4991-9a27-feb8c10c7235 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.680258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5007fd2fd5c745e4b08af70b4ebf162a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.681175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5007fd2fd5c745e4b08af70b4ebf162a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.684242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:41.685302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.685737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.686025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.686292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/4663046a4655413fb59199e7c254bdd4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.688729 ncdv-master-1 devstack@keystone.service[99132]: WARNING oslo_log.versionutils [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 24 21:11:41.689309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=4663046a4655413fb59199e7c254bdd4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.690584 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:41.690584 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:41.690584 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:41.695409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.705883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5007fd2fd5c745e4b08af70b4ebf162a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.730337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2796/5536] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:11:41 2022] GET /identity/v3/policies/4663046a4655413fb59199e7c254bdd4 => generated 236 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:41.731577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5007fd2fd5c745e4b08af70b4ebf162a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.733018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.733918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5007fd2fd5c745e4b08af70b4ebf162a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.742090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:41.772318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212cac9-f517-4701-8d43-dc0b317d0bd6 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.790908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2741/5537] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:41 2022] DELETE /identity/v3/users/5007fd2fd5c745e4b08af70b4ebf162a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:41.794605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.800120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:41.824063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.830671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.833061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.833363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/4663046a4655413fb59199e7c254bdd4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.837534 ncdv-master-1 devstack@keystone.service[99132]: WARNING oslo_log.versionutils [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 24 21:11:41.838373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=4663046a4655413fb59199e7c254bdd4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.840430 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:41.840430 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:41.840430 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:41.841872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.866831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2797/5538] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:41 2022] DELETE /identity/v3/policies/4663046a4655413fb59199e7c254bdd4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:41.872490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:41.896843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598598-affd-44fb-afca-08016c016c28 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.910402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ed1e8-9305-44b0-a5cf-ff818e8d7741 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:41.918850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.920159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.920461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.920802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/bc286f020d4440cc8e589bc43fd6ac9f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.923360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bc286f020d4440cc8e589bc43fd6ac9f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.925842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:41.990306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc286f020d4440cc8e589bc43fd6ac9f', 'request_id': 'req-12b0ebe6-c76b-4cbf-a886-f36fbd346659'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:41.992403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:41.993724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:41.994188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:41.994764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:41.997349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:41.997496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc286f020d4440cc8e589bc43fd6ac9f', 'request_id': 'req-12b0ebe6-c76b-4cbf-a886-f36fbd346659'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:42.010220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc286f020d4440cc8e589bc43fd6ac9f', 'request_id': 'req-12b0ebe6-c76b-4cbf-a886-f36fbd346659'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:42.010884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.031285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] The token cache is being invalidate because project bc286f020d4440cc8e589bc43fd6ac9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:42.031745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:42.032420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:42.034449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2742/5539] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:41 2022] DELETE /identity/v3/projects/bc286f020d4440cc8e589bc43fd6ac9f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:42.042342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.044071 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2798/5540] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:41 2022] POST /identity/v3/policies => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.044459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:42.045662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.048027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:42.064885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b0da-4749-4cf0-bf26-11ba2ffc8150 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.142326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:42.148440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0ebe6-c76b-4cbf-a886-f36fbd346659 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.157839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.158691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.159233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.159705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.161447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.161737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.162236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.162610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.163090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.164324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.164735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.166354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.201919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2799/5541] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/policies => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.203317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.216380 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2743/5542] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/projects => generated 359 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.218090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.218433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d1992-d1b4-4e3e-bedf-662d6af3ad7c tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.285336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:42.320418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdf807-0cdd-4064-9dcb-411eb9d7a984 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.328801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.329483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.329772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.330230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.330944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.333859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.336058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.337004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.337465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.337974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.339666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.341824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.365419 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2800/5543] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/policies => generated 230 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.369047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.380346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2744/5544] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/users => generated 417 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.382872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.386475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b4fc9-4a33-452f-96ae-49b4270c6fb2 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.403361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.404809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.406166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.406535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.407875 ncdv-master-1 devstack@keystone.service[99132]: WARNING oslo_log.versionutils [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. Sep 24 21:11:42.409165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.412232 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:42.412232 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:42.412232 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:42.415220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.425589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92edf3-d29f-4e10-ba22-2aec07410062 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.434620 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2801/5545] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:11:42 2022] GET /identity/v3/policies => generated 764 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:42.437742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.439356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.440836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.441729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.443215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.444072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.447206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.454840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf0529-c85a-452f-b477-304863401a61 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.462140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.462916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.463328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.463618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/68ba60146f0f4cdcbc60310428acc5b3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.464808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=68ba60146f0f4cdcbc60310428acc5b3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.466882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.472739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2745/5546] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/roles => generated 239 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.477381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.491053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2802/5547] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:42 2022] DELETE /identity/v3/policies/68ba60146f0f4cdcbc60310428acc5b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:42.493947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812216-d230-4f4c-8c51-f1c10cfb0de6 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.495107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.503037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.503884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.504291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.505582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/fec662a6f0c84496b21a524a3346a9e7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.510055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffa399-4553-4a0e-81d5-e765197b3d9f tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.541814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.542934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.542934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.543091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/bf4ed67e986d4fe2905b944dd90d7314` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.544093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=bf4ed67e986d4fe2905b944dd90d7314)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.545561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_user(user_id=fec662a6f0c84496b21a524a3346a9e7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.549263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.553101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.559802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2746/5548] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:11:42 2022] GET /identity/v3/users/fec662a6f0c84496b21a524a3346a9e7 => generated 417 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:42.560807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.580680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2803/5549] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:42 2022] DELETE /identity/v3/policies/bf4ed67e986d4fe2905b944dd90d7314 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:42.584931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8cacf-d50c-45fa-bb80-414cc68f2074 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.585459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.590303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.591606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.591606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.591820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/roles/ec7ece30912a447998ef33996236e2f6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.593540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_role(role_id=ec7ece30912a447998ef33996236e2f6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.594899 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:42.594899 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:42.594899 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:42.596100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.597417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b235e6-f770-4b1f-8352-247ef8391c51 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.602034 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2747/5550] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:11:42 2022] GET /identity/v3/roles/ec7ece30912a447998ef33996236e2f6 => generated 239 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:42.603133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.604470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd97d0bc50fed4f15af1029f6b52740f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfd69ea77d724d298f60f66b410c1b93', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.605330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.605679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.606050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] PATH_INFO: `/v3/policies/81d3b510907d443099b04ed7dbbb13e7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.607542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=81d3b510907d443099b04ed7dbbb13e7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.609651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.615386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22386b9-9c6a-4450-9a98-dca55321f0c7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.625699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.628472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.628987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.629293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.632061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2804/5551] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:42 2022] DELETE /identity/v3/policies/81d3b510907d443099b04ed7dbbb13e7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:42.632207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.637980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.642414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.644469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:42.707232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2748/5552] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:42 2022] POST /identity/v3/projects => generated 360 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:42.713260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.814331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14239938-beb8-4d2e-aedd-023d972b6777 tempest-PoliciesTestJSON-2052426528 tempest-PoliciesTestJSON-2052426528-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.887952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167819c-757c-450b-9735-81fc628918e5 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:42.894050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.895066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.896416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.896781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/fcfc07291f4f43b78ef18c4988302304` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.898751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_project(project_id=fcfc07291f4f43b78ef18c4988302304)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.902800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.912026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2749/5553] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:11:42 2022] GET /identity/v3/projects/fcfc07291f4f43b78ef18c4988302304 => generated 360 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:42.912026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:42.913191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:42.929928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:42.930548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:42.930836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:42.931108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] PATH_INFO: `/v3/users/f1e98565112a4b20b910b1bc8fa1a7e3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:42.943321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e98565112a4b20b910b1bc8fa1a7e3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:42.949806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:42.957622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:43.072341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:43.082236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05225762-7dc4-4d7a-94ea-2533d21a6e42 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:43.118481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:43.119432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.119867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.120676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] PATH_INFO: `/v3/projects/0340f96a146d4a3bb746713ae6ae6ebf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.122109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0340f96a146d4a3bb746713ae6ae6ebf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:43.125854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:43.143360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e98565112a4b20b910b1bc8fa1a7e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.143360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e98565112a4b20b910b1bc8fa1a7e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.162729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e98565112a4b20b910b1bc8fa1a7e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.191308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e98565112a4b20b910b1bc8fa1a7e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.192670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e98565112a4b20b910b1bc8fa1a7e3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.247769 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2805/5554] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:42 2022] DELETE /identity/v3/users/f1e98565112a4b20b910b1bc8fa1a7e3 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:43.248703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:43.253230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.256660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0340f96a146d4a3bb746713ae6ae6ebf', 'request_id': 'req-16293159-2ee7-40d8-aeef-86d28a559071'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.270769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0340f96a146d4a3bb746713ae6ae6ebf', 'request_id': 'req-16293159-2ee7-40d8-aeef-86d28a559071'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.285282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0340f96a146d4a3bb746713ae6ae6ebf', 'request_id': 'req-16293159-2ee7-40d8-aeef-86d28a559071'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.307229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] The token cache is being invalidate because project 0340f96a146d4a3bb746713ae6ae6ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:43.307582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.308452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.311414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2750/5555] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:42 2022] DELETE /identity/v3/projects/0340f96a146d4a3bb746713ae6ae6ebf => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:43.312417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:43.316604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.382349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb1609-c3cb-4ae9-baf2-944a27cce848 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:43.394591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:43.396023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.396336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.396622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/fcfc07291f4f43b78ef18c4988302304/users/fec662a6f0c84496b21a524a3346a9e7/roles/ec7ece30912a447998ef33996236e2f6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.414842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fcfc07291f4f43b78ef18c4988302304, user_id=fec662a6f0c84496b21a524a3346a9e7, role_id=ec7ece30912a447998ef33996236e2f6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:43.419628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:43.455960 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2806/5556] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:43 2022] PUT /identity/v3/projects/fcfc07291f4f43b78ef18c4988302304/users/fec662a6f0c84496b21a524a3346a9e7/roles/ec7ece30912a447998ef33996236e2f6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:43.462010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:43.462596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16293159-2ee7-40d8-aeef-86d28a559071 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:43.466334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.576849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aa2a3-c796-4fde-b93c-db3d07859d71 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:43.581696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:43.582727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.582902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.583104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] PATH_INFO: `/v3/users/27a03e9e5f7c43b4b6ef9433cf868ff1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.584310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a03e9e5f7c43b4b6ef9433cf868ff1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:43.586424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:43.597348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:43.622370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:43.625073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.625533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.626203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] PATH_INFO: `/v3/users/68adaeee5bc9449cb6776d67eace2906` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.633394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68adaeee5bc9449cb6776d67eace2906)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:43.641702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:43.657374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:43.666147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a03e9e5f7c43b4b6ef9433cf868ff1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.666956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a03e9e5f7c43b4b6ef9433cf868ff1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.678979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a03e9e5f7c43b4b6ef9433cf868ff1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.697909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a03e9e5f7c43b4b6ef9433cf868ff1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.699783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-af887d46-cfe9-44e5-99e4-5276ed25c6a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a03e9e5f7c43b4b6ef9433cf868ff1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.743865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2807/5557] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:43 2022] DELETE /identity/v3/users/27a03e9e5f7c43b4b6ef9433cf868ff1 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:43.747814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.749495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.749495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.762463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68adaeee5bc9449cb6776d67eace2906'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.763415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68adaeee5bc9449cb6776d67eace2906'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.785901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68adaeee5bc9449cb6776d67eace2906'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.787460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:43.788870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.795283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68adaeee5bc9449cb6776d67eace2906'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.796163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68adaeee5bc9449cb6776d67eace2906'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:43.824430 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2751/5558] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:43 2022] DELETE /identity/v3/users/68adaeee5bc9449cb6776d67eace2906 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:43.825619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:43.827699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.904604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2808/5559] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:43.905788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:43.915093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:43.950897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce94b-8b10-4008-a106-1eb3631fe35b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:43.958413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:43.959619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:43.960826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:43.961383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] PATH_INFO: `/v3/users/0ada02cf79d6453b90cd6ed4686dced7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:43.962969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ada02cf79d6453b90cd6ed4686dced7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:43.965717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.025051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52cb03-68a5-4b76-9752-a0a464e0e59d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.049203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ada02cf79d6453b90cd6ed4686dced7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.049902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ada02cf79d6453b90cd6ed4686dced7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.063513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ada02cf79d6453b90cd6ed4686dced7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.077140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ada02cf79d6453b90cd6ed4686dced7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.078129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40570300-04d7-414d-99d3-85d07da137c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ada02cf79d6453b90cd6ed4686dced7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.101225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:44.113487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.114787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.114787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.115518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.116207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.118217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8e3c1-6883-4ec8-bbed-90d30523a346 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.129342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2752/5560] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:43 2022] DELETE /identity/v3/users/0ada02cf79d6453b90cd6ed4686dced7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:44.129342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.129342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.129342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.137525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:44.154811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:44.156643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.167316 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2809/5561] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:43 2022] POST /identity/v3/projects => generated 358 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:44.169048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.169165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.169435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.202429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:44.204053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.250635 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2753/5562] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:44.250635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.253110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.305669 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2810/5563] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:44.307139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.309002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.339771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3abba-1720-4578-8ea8-94574e1ddcfc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.401582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95e08-a829-43d0-8314-f5d36c6b2667 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.408126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.408998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.409467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.409845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/5a9c09449f5947e6b01b3ac893839c73` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.411619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_project(project_id=5a9c09449f5947e6b01b3ac893839c73)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.414447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.420155 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2811/5564] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/projects/5a9c09449f5947e6b01b3ac893839c73 => generated 358 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:44.421482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.424982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.427152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.428174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.428678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.429475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.431169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.445963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.447552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.461146 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2754/5565] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:44.462716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.476788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0226-4106-410b-80ab-0f0facd0a7f8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.480972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.481673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.482028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.482381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.496080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.498583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.513291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2755/5566] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:44.514158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.526460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600446ff-9f78-43b6-892a-eecca87110fc service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.535191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.536837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.537356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.537707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.541257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b835599-66df-4757-9033-ed55e4d63d59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.546869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.547500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.547780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.548049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.551153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.552448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.555230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.569894 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2756/5567] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:44.570934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.572838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.575333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.587371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378fd35-505f-4fac-b67b-0dc41a2bec81 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.591625 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2812/5568] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:44.597852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.599865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.600006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.600248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/5a9c09449f5947e6b01b3ac893839c73/users/fec662a6f0c84496b21a524a3346a9e7/roles/ec7ece30912a447998ef33996236e2f6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.602486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_grant(project_id=5a9c09449f5947e6b01b3ac893839c73, user_id=fec662a6f0c84496b21a524a3346a9e7, role_id=ec7ece30912a447998ef33996236e2f6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.606048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.625376 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2757/5569] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:44 2022] PUT /identity/v3/projects/5a9c09449f5947e6b01b3ac893839c73/users/fec662a6f0c84496b21a524a3346a9e7/roles/ec7ece30912a447998ef33996236e2f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:44.630411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.640121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.641318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:44.656045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e237d7-dd3d-4f29-91d6-35b490625d30 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.682233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:44.700271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:44.831310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.832313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.833158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.833656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/fec662a6f0c84496b21a524a3346a9e7/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.835195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=fec662a6f0c84496b21a524a3346a9e7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.837733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.854828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cf905-9341-408f-a673-ab11aa11e40c tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.866918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.867566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.867848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.868168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] PATH_INFO: `/v3/projects/f39f22cf5bb04332a9c8579ff38d859c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.870066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f39f22cf5bb04332a9c8579ff38d859c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.872606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.905740 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2813/5570] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:11:44 2022] GET /identity/v3/users/fec662a6f0c84496b21a524a3346a9e7/projects => generated 842 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:44.907127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:44.922614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcca832-8dab-46e0-9f88-296d096f7650 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:44.929760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:44.930443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:44.931803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:44.932067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/5a9c09449f5947e6b01b3ac893839c73` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:44.933370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5a9c09449f5947e6b01b3ac893839c73)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:44.935753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:44.952334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f39f22cf5bb04332a9c8579ff38d859c', 'request_id': 'req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.960530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f39f22cf5bb04332a9c8579ff38d859c', 'request_id': 'req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.975779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f39f22cf5bb04332a9c8579ff38d859c', 'request_id': 'req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.994530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] The token cache is being invalidate because project f39f22cf5bb04332a9c8579ff38d859c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:44.995118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.996110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f589ea3b-b3a9-44dd-8715-33c1e8a9c542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:44.999004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2758/5571] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:44 2022] DELETE /identity/v3/projects/f39f22cf5bb04332a9c8579ff38d859c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.003927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.004641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.005283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.038330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:45.040669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9c09449f5947e6b01b3ac893839c73', 'request_id': 'req-9285c98d-6971-43b9-ad44-529a775d6bd9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.047046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.050419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9c09449f5947e6b01b3ac893839c73', 'request_id': 'req-9285c98d-6971-43b9-ad44-529a775d6bd9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.062484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9c09449f5947e6b01b3ac893839c73', 'request_id': 'req-9285c98d-6971-43b9-ad44-529a775d6bd9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.085101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] The token cache is being invalidate because project 5a9c09449f5947e6b01b3ac893839c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.085566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.086544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.088680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2814/5572] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:44 2022] DELETE /identity/v3/projects/5a9c09449f5947e6b01b3ac893839c73 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.089679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.091655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.147307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2759/5573] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:45.148939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.150921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.184619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285c98d-6971-43b9-ad44-529a775d6bd9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.190201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.190863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.191141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.191413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] PATH_INFO: `/v3/projects/cab8f6881ee24233aa25f3df3f2876e1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.194574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab8f6881ee24233aa25f3df3f2876e1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.196504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.208410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:45.215652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:45.237735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d29f9-8c64-4eaf-b593-e45ba73c7f92 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.245946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.246777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.247674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.247674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/fcfc07291f4f43b78ef18c4988302304` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.248663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fcfc07291f4f43b78ef18c4988302304)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.251668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.262131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab8f6881ee24233aa25f3df3f2876e1', 'request_id': 'req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.268667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab8f6881ee24233aa25f3df3f2876e1', 'request_id': 'req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.276394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab8f6881ee24233aa25f3df3f2876e1', 'request_id': 'req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.295315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] The token cache is being invalidate because project cab8f6881ee24233aa25f3df3f2876e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.295776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.296535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a06a5c82-96fa-4198-9c67-3f6eafdfbbe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.300903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2815/5574] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/projects/cab8f6881ee24233aa25f3df3f2876e1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.300903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.306850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.307582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.312776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfc07291f4f43b78ef18c4988302304', 'request_id': 'req-6738e192-c3c2-4100-b240-18776d6e22bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.320147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfc07291f4f43b78ef18c4988302304', 'request_id': 'req-6738e192-c3c2-4100-b240-18776d6e22bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.344601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:45.345999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfc07291f4f43b78ef18c4988302304', 'request_id': 'req-6738e192-c3c2-4100-b240-18776d6e22bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.346447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.364817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] The token cache is being invalidate because project fcfc07291f4f43b78ef18c4988302304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.365035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.365763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6738e192-c3c2-4100-b240-18776d6e22bd tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.370545 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2760/5575] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/projects/fcfc07291f4f43b78ef18c4988302304 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.370545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.370545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.370960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.406375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:45.408190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.464021 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2816/5576] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:45.465163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.467422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.517392 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2761/5577] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:45.518455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.522237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.535620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c3df1-232b-4327-93e6-52f716f15338 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.556855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.557849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.558391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.558958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] PATH_INFO: `/v3/projects/b1ccbc91c8f0461b8a3de1250cbecfbd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.561163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ccbc91c8f0461b8a3de1250cbecfbd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.563695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.593078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c452db-6c7f-4deb-ae90-f9ccd4e3bb61 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.599805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.600563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.600947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.601411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/roles/ec7ece30912a447998ef33996236e2f6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.604333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_role(role_id=ec7ece30912a447998ef33996236e2f6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.606434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.628080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ccbc91c8f0461b8a3de1250cbecfbd', 'request_id': 'req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.643171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ccbc91c8f0461b8a3de1250cbecfbd', 'request_id': 'req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.663845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ccbc91c8f0461b8a3de1250cbecfbd', 'request_id': 'req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.677721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invalidating the token cache because role ec7ece30912a447998ef33996236e2f6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.678077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.678841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.680722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2817/5578] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/roles/ec7ece30912a447998ef33996236e2f6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.681572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.683209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.689101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:45.693909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:45.700549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] The token cache is being invalidate because project b1ccbc91c8f0461b8a3de1250cbecfbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.700957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.701863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ef5e1e53-2d8e-49d6-8834-cac2e1df628a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.707396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2762/5579] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/projects/b1ccbc91c8f0461b8a3de1250cbecfbd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.709926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.710320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.710772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.733746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:45.736273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.799838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fefd34-b90d-4a1a-8fd6-dd7f154a4ed9 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.808000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.808683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.808962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.809228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.809873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:45.810172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.811823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.834756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2818/5580] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:45.835789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.838458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.842823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2763/5581] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:45.842823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:45.843654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:45.857482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a7109-cbeb-4c0f-8eeb-1acd269bee12 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.866359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:45.867120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.867577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.867729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] PATH_INFO: `/v3/projects/c78df8495b6849d58b71a6ccbf5041ee` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:45.869136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78df8495b6849d58b71a6ccbf5041ee)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:45.872447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:45.941056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78df8495b6849d58b71a6ccbf5041ee', 'request_id': 'req-224978ae-a236-4970-b512-4365804f4551'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.948507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78df8495b6849d58b71a6ccbf5041ee', 'request_id': 'req-224978ae-a236-4970-b512-4365804f4551'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.956728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78df8495b6849d58b71a6ccbf5041ee', 'request_id': 'req-224978ae-a236-4970-b512-4365804f4551'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.956904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84999b-4bf7-4694-b480-f4c26d346a95 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:45.971170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] The token cache is being invalidate because project c78df8495b6849d58b71a6ccbf5041ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:45.971459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.972284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-224978ae-a236-4970-b512-4365804f4551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:45.975093 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2819/5582] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/projects/c78df8495b6849d58b71a6ccbf5041ee => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:45.976358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:45.977257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:45.977548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.000846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:46.001991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.030960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.031618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.032490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.032778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/fec662a6f0c84496b21a524a3346a9e7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.034433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_user(user_id=fec662a6f0c84496b21a524a3346a9e7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.037199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.071295 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2820/5583] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:46.072308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.073713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.086238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14d8fd-e1ff-4f25-a79b-0cb90d6f6061 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.091191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.091640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.091909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.092184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.092930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.094898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.104413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec662a6f0c84496b21a524a3346a9e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.105644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec662a6f0c84496b21a524a3346a9e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.109976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:46.116550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec662a6f0c84496b21a524a3346a9e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.119592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:46.127660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec662a6f0c84496b21a524a3346a9e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.128611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec662a6f0c84496b21a524a3346a9e7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.135466 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2821/5584] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:46 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:46.137530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.140669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.149754 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2764/5585] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:45 2022] DELETE /identity/v3/users/fec662a6f0c84496b21a524a3346a9e7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:46.150457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.151794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.231956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394dfa7-a1c2-4e0a-973e-d5410fe209bc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.239437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dd281-009b-4ac3-bd25-f80f182abd87 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.241655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.243001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.243527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.243902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.244703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:46.245085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.245314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.245761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.246050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.246436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] PATH_INFO: `/v3/users/d97d0bc50fed4f15af1029f6b52740f2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.247756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e238e1-e7aa-4c0c-935f-728775a7c038 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.247879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d97d0bc50fed4f15af1029f6b52740f2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.249916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.265165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2822/5586] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:46.265165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.265165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.265165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.290851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:46.292160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.306337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2823/5587] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:46.306337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.308728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.317766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd97d0bc50fed4f15af1029f6b52740f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.319306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd97d0bc50fed4f15af1029f6b52740f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.325540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f1454-6de7-48cc-8146-7b47f4501057 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.329928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd97d0bc50fed4f15af1029f6b52740f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.332096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.333490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.334055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.334425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.335184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.338073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.339888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd97d0bc50fed4f15af1029f6b52740f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.340751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd97d0bc50fed4f15af1029f6b52740f2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.368542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2824/5588] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:46 2022] POST /identity/v3/users => generated 294 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:46.369642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.373768 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2765/5589] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:46 2022] DELETE /identity/v3/users/d97d0bc50fed4f15af1029f6b52740f2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:46.373768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.384160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.386625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.505314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:46.550331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a498040-ada0-4c78-8507-c07ad94f5ad8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.550621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fba8fd-aa59-456a-ace1-f86ee94da366 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.579985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.580658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.580943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.581226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.582046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.584872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.634196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2766/5590] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:46 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:46.634196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.640408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.717003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.717823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.718236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.718390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/79408afb96b54acbbd4445328684f6c7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.721104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=79408afb96b54acbbd4445328684f6c7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.724828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.781652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:46.826568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75302715-6f9c-4523-9b77-1c844e6fa490 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.840777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:46.841475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:46.841681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:46.841953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:46.842646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:46.843344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:46.851573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:46.876172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79408afb96b54acbbd4445328684f6c7', 'request_id': 'req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.881345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2767/5591] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:46.882448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.887663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:46.894432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79408afb96b54acbbd4445328684f6c7', 'request_id': 'req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.922107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c35591-df11-4535-9edf-a6959a3a57a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:46.929908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79408afb96b54acbbd4445328684f6c7', 'request_id': 'req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.960039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] The token cache is being invalidate because project 79408afb96b54acbbd4445328684f6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:46.960039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.960944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:46.964057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2825/5592] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:46 2022] DELETE /identity/v3/projects/79408afb96b54acbbd4445328684f6c7 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:46.965174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:46.969992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.053515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.056432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.056432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.057003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.058286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.060353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.077907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3bf33-d231-4c65-aae5-8a9c01a8fdca tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.081042 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2768/5593] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:47.081355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.084373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.092309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.093107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.093541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.093951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.094802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.096866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.126648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2826/5594] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:46 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:47.127071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.128844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.142691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f462b0f-da56-4d40-8b31-1b70239d3b41 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.149527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.151096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.151096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.151374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.152335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.155227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.166800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:47.201976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc771d-afcc-4206-9bd8-3dad66e54660 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.202739 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2827/5595] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:47 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:47.203696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.205498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.285167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a2ca2-7d6c-431e-9eb8-b124316a63d2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.291327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.291977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.292256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.292685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] PATH_INFO: `/v3/projects/bfd69ea77d724d298f60f66b410c1b93` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.294030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd69ea77d724d298f60f66b410c1b93)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.296001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.300879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.301916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.302391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.302706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.304508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.322810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:47.338117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.339908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.370202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2769/5596] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:47.371157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.373067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.376946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd69ea77d724d298f60f66b410c1b93', 'request_id': 'req-b6a77268-50d1-42e3-a60b-f299d11e7c14'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.385586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd69ea77d724d298f60f66b410c1b93', 'request_id': 'req-b6a77268-50d1-42e3-a60b-f299d11e7c14'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.400302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd69ea77d724d298f60f66b410c1b93', 'request_id': 'req-b6a77268-50d1-42e3-a60b-f299d11e7c14'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.431410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] The token cache is being invalidate because project bfd69ea77d724d298f60f66b410c1b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:47.431710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.432409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.434439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2828/5597] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:47 2022] DELETE /identity/v3/projects/bfd69ea77d724d298f60f66b410c1b93 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:47.440806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.443231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.527815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f31c7d-c00e-4036-aa32-7cc92ab4e750 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.600898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a77268-50d1-42e3-a60b-f299d11e7c14 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.614836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.615480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.615753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.616170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.617991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.620166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.636968 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2829/5598] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:47.636968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.650554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.667371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6fc3f-b353-4e98-95fc-78fd487a9005 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.694860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.695739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.698894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.699280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.700291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.704506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.705173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.705457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.705735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/395c85cb5dff4c31a371d2d39fe594a2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.706470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.713071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:update_user(user_id=395c85cb5dff4c31a371d2d39fe594a2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.714128 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:47.714128 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:11:47.714128 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:47.715478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.747875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2830/5599] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:47.750447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.763728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.787197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.789623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57353a63-5852-4103-ae7b-d93b8ed1acf9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.800200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.800920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.801068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.806003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.807109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:47.807210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.810248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:47.820419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invalidating the token cache because user 395c85cb5dff4c31a371d2d39fe594a2 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:47.820727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.821957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de991127-4f42-4d64-a859-17a62d39f710 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:47.823862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2770/5600] 10.222.0.52 () {60 vars in 1243 bytes} [Sat Sep 24 21:11:47 2022] PATCH /identity/v3/users/395c85cb5dff4c31a371d2d39fe594a2 => generated 297 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:47.825099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57401e-b388-496b-af01-5de55f713169 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.825382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57401e-b388-496b-af01-5de55f713169 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.825655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57401e-b388-496b-af01-5de55f713169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.871002 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2831/5601] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:47 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:47.873461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.884876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a57401e-b388-496b-af01-5de55f713169 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:47.885985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.890793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a57401e-b388-496b-af01-5de55f713169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.911048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6d87e-d44e-4ccd-8c0c-3e3dbe04b16d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.922128 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2771/5602] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:47.923304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57401e-b388-496b-af01-5de55f713169 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:47.930098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a57401e-b388-496b-af01-5de55f713169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:47.940360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.941185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.941601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.942174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] PATH_INFO: `/v3/projects/2320b653932749b2bba237f24235cbf1/users/72343eb12cea4915b2ae7fd342fb6ab1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.964220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57401e-b388-496b-af01-5de55f713169 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:47.988462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:47.990116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:47.990435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:47.991776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:47.992455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:47.994079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43272aec-8069-4a72-a54e-38c412c5fc1c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.005087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2320b653932749b2bba237f24235cbf1, user_id=72343eb12cea4915b2ae7fd342fb6ab1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.009052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.029590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2772/5603] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:48.031223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7708-a33e-4324-9636-3b405eda48fb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.031514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7708-a33e-4324-9636-3b405eda48fb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.031805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7708-a33e-4324-9636-3b405eda48fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.058547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2832/5604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:47 2022] PUT /identity/v3/projects/2320b653932749b2bba237f24235cbf1/users/72343eb12cea4915b2ae7fd342fb6ab1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:48.058547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.058547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.102770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83dd7708-a33e-4324-9636-3b405eda48fb None None] MFA Rules not processed for user `395c85cb5dff4c31a371d2d39fe594a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:48.103153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83dd7708-a33e-4324-9636-3b405eda48fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.111119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2773/5605] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:48 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:11:48.112708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.113001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.114082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.142404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:48.144596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.166218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:48.203474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be14ef-3adb-4083-ac30-e82b123de654 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.216846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.217619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.217910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.218337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.219034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.222581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.255852 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2833/5606] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:48.255852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.288995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412783f-73aa-43fe-9d78-a59aa365e15f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.304502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.308264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.308594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.308896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] PATH_INFO: `/v3/projects/77274d03cfb74d738b3b75c92e4cd641/users/37fee5c58bad45aca56e2bb3b7a3af85/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.315119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2774/5607] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:48.320164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.321522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.356601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77274d03cfb74d738b3b75c92e4cd641, user_id=37fee5c58bad45aca56e2bb3b7a3af85, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.359501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13dab4-f9bc-4cd2-8663-56c5f682d561 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.393650 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2834/5608] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:48 2022] PUT /identity/v3/projects/77274d03cfb74d738b3b75c92e4cd641/users/37fee5c58bad45aca56e2bb3b7a3af85/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:48.401303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.401754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.402105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.435558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6373b-9e49-4270-9e8e-0ec7aeaf06af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.450543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] MFA Rules not processed for user `72343eb12cea4915b2ae7fd342fb6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:48.452098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.523722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.525107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.525594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.526372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/395c85cb5dff4c31a371d2d39fe594a2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.529234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:update_user(user_id=395c85cb5dff4c31a371d2d39fe594a2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.532978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.560253 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2835/5609] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:48.561259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.591350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:48.604493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:48.626589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invalidating the token cache because user 395c85cb5dff4c31a371d2d39fe594a2 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:48.627207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:48.628098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:48.632703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2775/5610] 10.222.0.52 () {60 vars in 1243 bytes} [Sat Sep 24 21:11:48 2022] PATCH /identity/v3/users/395c85cb5dff4c31a371d2d39fe594a2 => generated 297 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:48.633907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.636736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.678631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f92a5-b8bf-4e09-a57a-9d844a1ba6c7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.697518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.697518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.697518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.697821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.698372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.700063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e2ac1-c026-46a4-a9d1-85ac9fd94cdc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.717131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2836/5611] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:48.721375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.721659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.722537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.743708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:48.745038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.755849 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2837/5612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:48.757160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.758557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.769169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3983f8-42b2-4ba9-afaa-6081e5a32a89 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.774352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d9355-bca5-417a-9c02-4110fab58877 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.777669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.778396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.778694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.779057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.781475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:48.781475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.782395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.786629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.787787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.788309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.788767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.789647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.792500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.807945 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2776/5613] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:48.809059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.811327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.840691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6cee7-d185-4be1-b20f-24bb210e8f79 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.843089 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2838/5614] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:48 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:48.844038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:48.846361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:48.871446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:48.924520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.925473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.925818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.926718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/395c85cb5dff4c31a371d2d39fe594a2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.932624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5dcf2-4b39-4fab-bd30-a855979e7921 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:48.938913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:48.940029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:48.940375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:48.941255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] PATH_INFO: `/v3/projects/b97081b61e424f0ba6722a4f7655f0d0/users/a86feee574c341aa9adba8b50219d5fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:48.943735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:update_user(user_id=395c85cb5dff4c31a371d2d39fe594a2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.946199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.960142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97081b61e424f0ba6722a4f7655f0d0, user_id=a86feee574c341aa9adba8b50219d5fc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:48.963693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:48.987091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:48.997791 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2839/5615] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:48 2022] PUT /identity/v3/projects/b97081b61e424f0ba6722a4f7655f0d0/users/a86feee574c341aa9adba8b50219d5fc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:48.997791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.000135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.004451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invalidating the token cache because user 395c85cb5dff4c31a371d2d39fe594a2 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:49.004912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:49.005935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:49.008186 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2777/5616] 10.222.0.52 () {60 vars in 1243 bytes} [Sat Sep 24 21:11:48 2022] PATCH /identity/v3/users/395c85cb5dff4c31a371d2d39fe594a2 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:49.009033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.010394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.025528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78687da2-a1d3-4b26-a565-18b75be40a15 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.077129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:49.088521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bd998-31f8-4de2-b66c-d8dd57e916a6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.092533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.093130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.093409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.093675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.096017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.098431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.099380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.099748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.100025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.100655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.102767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426845-9cf2-43ff-b563-e4ee590618da admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.110876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.112335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.124683 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2840/5617] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:49.125298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.126779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.141530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6c73-83b5-4711-a81b-8fb2d07a7f4c service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.149800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2778/5618] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:49.149800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.149800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.149800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.150706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.151149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.151434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.151706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.152341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.154521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c4371-c562-4747-9d1f-25b5aceb197e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.187312 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2841/5619] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/users => generated 299 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:49.189144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.189692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.190218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.200776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] MFA Rules not processed for user `a86feee574c341aa9adba8b50219d5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:49.202804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.230479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] MFA Rules not processed for user `395c85cb5dff4c31a371d2d39fe594a2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:49.231727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.235820 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2842/5620] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/auth/tokens => generated 349 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:11:49.236750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.334670 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2779/5621] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:49.335934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.378574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42272e-e7a8-4a24-bc62-4a19066f3745 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.402529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.402529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.402529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.402529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.403279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.403279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.427530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:49.448533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2843/5622] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:49.449934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.452956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.568180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63309d-fac8-4859-a111-5eb09fd77de2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.577973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.578803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa730f-041a-4c81-b25c-ce70fe9914ea None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.579190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.584219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.584382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.584694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/395c85cb5dff4c31a371d2d39fe594a2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.584814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.585091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.585362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.586026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.586281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:update_user(user_id=395c85cb5dff4c31a371d2d39fe594a2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.587946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.593556 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:11:49.593556 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:11:49.593556 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:49.595321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.602534 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2780/5623] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:49.603980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.606928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.688186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:49.702523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invalidating the token cache because user 395c85cb5dff4c31a371d2d39fe594a2 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:49.702523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:49.702523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:49.708401 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2844/5624] 10.222.0.52 () {60 vars in 1243 bytes} [Sat Sep 24 21:11:49 2022] PATCH /identity/v3/users/395c85cb5dff4c31a371d2d39fe594a2 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:49.710806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.712440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.717693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cda45-0945-4dc1-88fb-ff3b1daee449 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.727849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.728432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.728782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.729290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.731476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.740234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5312562-153b-4e52-9027-ab9903daf409 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.749322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.751207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.756319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.757478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.759322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.760344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.761285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:49.763910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.772177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.780476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2781/5625] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:49.781189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.783140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.799865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2845/5626] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:49.801545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.804078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.824313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b284c1-78f7-4238-a41a-a5141b6526db service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.825250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ce551-4ed0-407c-9ecc-c16c52dcd312 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.834215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.835429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.836530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.837234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.838880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.842930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.846094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:49.852907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:49.853206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:49.853607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] PATH_INFO: `/v3/projects/3e2db6b37f824828a46ca070a7b775a7/users/5da4e78ed8ad4a7fab639e0f0475c8ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:49.898193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2846/5627] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:11:49 2022] POST /identity/v3/users => generated 264 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:49.898551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2db6b37f824828a46ca070a7b775a7, user_id=5da4e78ed8ad4a7fab639e0f0475c8ad, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:49.899380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.901977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:49.904697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:49.921553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ebad6-2c9b-495a-8161-b8e070efada7 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:49.929805 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2782/5628] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:49 2022] PUT /identity/v3/projects/3e2db6b37f824828a46ca070a7b775a7/users/5da4e78ed8ad4a7fab639e0f0475c8ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:49.929805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:49.999884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.000718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.001002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.001277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] PATH_INFO: `/v3/users/a86feee574c341aa9adba8b50219d5fc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.002615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86feee574c341aa9adba8b50219d5fc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.006355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.027265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb25e87-347c-40c9-99b9-d7c94b5ec751 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.033143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.033984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.034451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.035206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.036487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.038440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.051983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2783/5629] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:50.052800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.093324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:UPDATE=1,keystone:INSERT=5,keystone:DELETE=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:50.098766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86feee574c341aa9adba8b50219d5fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.099809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86feee574c341aa9adba8b50219d5fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.116082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86feee574c341aa9adba8b50219d5fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.134320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86feee574c341aa9adba8b50219d5fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.135818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86feee574c341aa9adba8b50219d5fc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.152692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b4c6-a034-4c2e-903a-258c57059cd9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.158474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.159621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.160196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.161303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/b9143d246cb844f2821a0e4aad294b26/password` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.191910 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2847/5630] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:49 2022] DELETE /identity/v3/users/a86feee574c341aa9adba8b50219d5fc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:50.191910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.191910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.247991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:50.260950 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2784/5631] 10.222.0.52 () {60 vars in 1269 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/users/b9143d246cb844f2821a0e4aad294b26/password => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:50.261923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.265751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.290843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50e92e-4eb1-4773-b512-6ba2871e6a59 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.319650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876da88-ae1f-4f26-9075-b8d3a3ec878e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.322128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:50.328986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.331682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.333771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.334265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.335426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.339895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2f882-712e-4044-9cf3-d987d8991ee3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.361040 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2848/5632] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:50.363917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16b954-9fd4-4f0e-8677-6fdbaab28794 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.364414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16b954-9fd4-4f0e-8677-6fdbaab28794 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.364847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16b954-9fd4-4f0e-8677-6fdbaab28794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.369860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.370702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.371086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.371490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] PATH_INFO: `/v3/projects/dab1f00857ac4d9e849e6f86a0ae15ce/users/a0c27ba4af2b4ac58754e4e13625016c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.396008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7b16b954-9fd4-4f0e-8677-6fdbaab28794 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:50.397252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7b16b954-9fd4-4f0e-8677-6fdbaab28794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.397562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab1f00857ac4d9e849e6f86a0ae15ce, user_id=a0c27ba4af2b4ac58754e4e13625016c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.402084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.413535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2849/5633] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:50.414948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f9281-5646-4196-b238-3c6fdd6c8aa1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.415232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f9281-5646-4196-b238-3c6fdd6c8aa1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.415527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f9281-5646-4196-b238-3c6fdd6c8aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.429513 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2785/5634] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:50 2022] PUT /identity/v3/projects/dab1f00857ac4d9e849e6f86a0ae15ce/users/a0c27ba4af2b4ac58754e4e13625016c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:50.430582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.481082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c46f9281-5646-4196-b238-3c6fdd6c8aa1 None None] MFA Rules not processed for user `37fee5c58bad45aca56e2bb3b7a3af85`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:50.482766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c46f9281-5646-4196-b238-3c6fdd6c8aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.525873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854854df-af1a-4736-adc0-8e93d07fbc9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.532713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.533473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.533743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.534023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] PATH_INFO: `/v3/projects/3e2db6b37f824828a46ca070a7b775a7/users/5da4e78ed8ad4a7fab639e0f0475c8ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.537029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2db6b37f824828a46ca070a7b775a7, user_id=5da4e78ed8ad4a7fab639e0f0475c8ad, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.541093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.563551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2786/5635] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:50 2022] PUT /identity/v3/projects/3e2db6b37f824828a46ca070a7b775a7/users/5da4e78ed8ad4a7fab639e0f0475c8ad/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:50.563551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.563551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.591842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2850/5636] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:50.593238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201eefe-76b8-4432-bf26-37578d914868 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.593585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201eefe-76b8-4432-bf26-37578d914868 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.594098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201eefe-76b8-4432-bf26-37578d914868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.638402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0201eefe-76b8-4432-bf26-37578d914868 None None] MFA Rules not processed for user `a0c27ba4af2b4ac58754e4e13625016c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:50.639883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0201eefe-76b8-4432-bf26-37578d914868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.657162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d487f-6804-4199-879b-a58111bc80e2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.661933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.662801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.663127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.663404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.684355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:50.731265 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2851/5637] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:50.733439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.733439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.733439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.743139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:50.745533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.747095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.758930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2787/5638] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:50.759743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.761058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.764248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] MFA Rules not processed for user `5da4e78ed8ad4a7fab639e0f0475c8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:50.765167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:50.775360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51322a-b8f4-40d5-84fc-11dd44a2633b service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.781297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.782005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.782147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.782886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.783126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.786960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.821800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2788/5639] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:50.822649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.908728 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2852/5640] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:50.910182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a62fad-d322-4517-94e3-847166c0e127 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:50.910430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.918396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0c27ba4af2b4ac58754e4e13625016c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dab1f00857ac4d9e849e6f86a0ae15ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:50.919195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:50.919526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:50.919801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] PATH_INFO: `/v3/auth/catalog` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:50.920483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:50.922403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:50.924253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2789/5641] 10.222.0.52 () {58 vars in 1143 bytes} [Sat Sep 24 21:11:50 2022] GET /identity/v3/auth/catalog => generated 2658 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:50.925094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:50.926185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239801-bcc4-40eb-8919-913e21234137 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.024804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.025435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.026619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.026619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.026619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.028683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.056781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407478a-6505-4b60-8072-d8963cad72e6 tempest-IdentityCatalogTest-1167291721 tempest-IdentityCatalogTest-1167291721-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.061204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.061814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.062212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.062523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.063331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.065032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.096177 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2853/5642] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/users => generated 309 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:51.097400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.118716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ef49-c1ce-4856-a5f5-f84292d6a4ea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.128413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.130194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.131201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.131761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] PATH_INFO: `/v3/users/a0c27ba4af2b4ac58754e4e13625016c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.133155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c27ba4af2b4ac58754e4e13625016c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.137540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.146783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2790/5643] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:50 2022] POST /identity/v3/projects => generated 360 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:51.146783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.206414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:51.214396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c27ba4af2b4ac58754e4e13625016c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.215758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c27ba4af2b4ac58754e4e13625016c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.227126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c27ba4af2b4ac58754e4e13625016c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.243116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c27ba4af2b4ac58754e4e13625016c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.243888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c27ba4af2b4ac58754e4e13625016c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.246738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1cc-ea13-4c91-a081-30de9666e00b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.257010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.260108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2854/5644] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:51 2022] DELETE /identity/v3/users/a0c27ba4af2b4ac58754e4e13625016c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:51.260269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.260269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.260269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.260531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.260625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.260719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.261364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.277655 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2791/5645] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:51.278745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.281882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.294379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:51.346958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea155c43-f165-4b0b-bb74-f0e81b8e7dc0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.355435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.356093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.356373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.356645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] PATH_INFO: `/v3/projects/b97081b61e424f0ba6722a4f7655f0d0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.357974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97081b61e424f0ba6722a4f7655f0d0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.361694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.385787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164980e8-21a0-4b0d-b280-ed43a7192ec4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.423087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97081b61e424f0ba6722a4f7655f0d0', 'request_id': 'req-ca86eb20-ffe1-4870-8920-6d7f4d110113'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.429921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97081b61e424f0ba6722a4f7655f0d0', 'request_id': 'req-ca86eb20-ffe1-4870-8920-6d7f4d110113'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.436887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97081b61e424f0ba6722a4f7655f0d0', 'request_id': 'req-ca86eb20-ffe1-4870-8920-6d7f4d110113'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.453835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] The token cache is being invalidate because project b97081b61e424f0ba6722a4f7655f0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:51.454080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.455943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca86eb20-ffe1-4870-8920-6d7f4d110113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:51.458236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2855/5646] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:51 2022] DELETE /identity/v3/projects/b97081b61e424f0ba6722a4f7655f0d0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:51.459681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e6c7-2c56-4499-abdb-4ccbdd86ce2e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.460077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e6c7-2c56-4499-abdb-4ccbdd86ce2e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.460539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9e6c7-2c56-4499-abdb-4ccbdd86ce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.464164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.465073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.465512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.465938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1454227724` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.467359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=5af2e76deb7c4f54bd423f06ef9f793c, value=tempest-tag-1454227724)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.470373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.487479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a4b9e6c7-2c56-4499-abdb-4ccbdd86ce2e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:51.489536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4b9e6c7-2c56-4499-abdb-4ccbdd86ce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.525799 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-20c1c47b-14b6-49d5-832b-6807c799cdb5 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1454227724. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:11:51.525799 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:11:51.525799 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:11:51.537594 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2792/5647] 10.222.0.52 () {60 vars in 1333 bytes} [Sat Sep 24 21:11:51 2022] PUT /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1454227724 => generated 0 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 291 bytes (1 switches on core 0) Sep 24 21:11:51.539393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.539815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.540188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.549103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:51.586817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] MFA Rules not processed for user `b9143d246cb844f2821a0e4aad294b26`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:51.588280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.597269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2793/5648] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:51 2022] POST /identity/v3/auth/tokens => generated 329 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:11:51.599020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.600440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.614026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2856/5649] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:51.615842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.616139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.616469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.632137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc599fa-e82d-49fe-b39f-d8480e96f8e5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.658395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.659193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.659472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.659740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] PATH_INFO: `/v3/projects/26ca384451ad438eb7d7e074fd11a122/users/b2980f6962204dd6a16c5f7d1ed2e127/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.672727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:51.674292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.688084 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2857/5650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:51.688955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.690396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.704709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ca384451ad438eb7d7e074fd11a122, user_id=b2980f6962204dd6a16c5f7d1ed2e127, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.708179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.734955 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2794/5651] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:51 2022] PUT /identity/v3/projects/26ca384451ad438eb7d7e074fd11a122/users/b2980f6962204dd6a16c5f7d1ed2e127/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:51.734955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.743682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.817305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570c508-8022-4b6b-ac78-677c36ec992a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.842862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:51.899656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732d84-8604-433b-9689-db94a93268d1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:51.909989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.912697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.912980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.913256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.935856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:51.936533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.936814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.937110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1454227724` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:51.937460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.939917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.951347 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2795/5652] 10.222.0.52 () {56 vars in 1263 bytes} [Sat Sep 24 21:11:51 2022] GET /identity/v3/auth/tokens => generated 329 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 21:11:51.953685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:51.955367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:51.955509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=5af2e76deb7c4f54bd423f06ef9f793c, value=tempest-tag-1454227724)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:51.958477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13794516-c091-4608-a573-b8cf9fa04506 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:51.986884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2858/5653] 10.222.0.52 () {58 vars in 1314 bytes} [Sat Sep 24 21:11:51 2022] GET /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1454227724 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:51.994713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:51.995036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:51.995280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.052862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:52.057449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:52.108894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ec5c-2a47-402e-b175-6f4b9eb9644e tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.121566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.122877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.123163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.123437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.183127 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2859/5654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:52.183127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.201045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65646033-7966-42ab-aed8-b76c4f042912 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.216804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.217435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.217714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.218435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.218662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.220750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.237395 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2860/5655] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:52.237395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.243278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:52.249881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5ada3-ef9f-4b8b-97c7-71fa7dee5192 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.256324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.257079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.257488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.257835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/b9143d246cb844f2821a0e4aad294b26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.259088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b9143d246cb844f2821a0e4aad294b26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.260735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.261453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.262210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.285671 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2796/5656] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:52.297975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.324344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe1f98-f3bf-48c8-b42c-fed2c88d056c service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.335571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.338960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.339474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.340764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.343474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=5af2e76deb7c4f54bd423f06ef9f793c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.347994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:52.349328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.350293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:52.365488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:52.399075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:52.399929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9143d246cb844f2821a0e4aad294b26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:52.424387 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2861/5657] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:52 2022] DELETE /identity/v3/users/b9143d246cb844f2821a0e4aad294b26 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:52.424387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.425617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:52.426607 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2797/5658] 10.222.0.52 () {60 vars in 1265 bytes} [Sat Sep 24 21:11:52 2022] PUT /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags => generated 140 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:52.430072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.433919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:52.466142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e287-6185-46d8-9b58-25cc612ceda1 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.555780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:52.587694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d91eab-258a-429d-ae32-7fe27dae6e64 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.593920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.595014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.595459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.595894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.596846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:52.597425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.599989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.602225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.602916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.603199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.603478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] PATH_INFO: `/v3/projects/26ca384451ad438eb7d7e074fd11a122/users/b2980f6962204dd6a16c5f7d1ed2e127/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.615124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ca384451ad438eb7d7e074fd11a122, user_id=b2980f6962204dd6a16c5f7d1ed2e127, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.617103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2862/5659] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:52.618160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.620564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:52.621425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.644211 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2798/5660] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:52 2022] PUT /identity/v3/projects/26ca384451ad438eb7d7e074fd11a122/users/b2980f6962204dd6a16c5f7d1ed2e127/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:52.645119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.662334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:52.765219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce735910-8905-42c1-9a9c-30ad1eda082b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.789539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:52.811648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64e938-4e41-4de2-89f5-609d91589099 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.817959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.820076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.820728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.821353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.841018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=5af2e76deb7c4f54bd423f06ef9f793c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.844260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.851096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.852254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.852804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.853225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.854424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.856750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.861535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2799/5661] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Sep 24 21:11:52 2022] GET /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags => generated 140 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:52.862772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.904562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2863/5662] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:52 2022] POST /identity/v3/users => generated 364 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:52.906294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:52.933398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3764dc-a2dd-4bf7-b18c-58a8dc4771e1 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.946406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.947402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.949932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.950235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1531097391` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.953057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2509-d92e-481d-bfca-142d0b11db43 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:52.956064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=5af2e76deb7c4f54bd423f06ef9f793c, value=tempest-tag-1531097391)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.959896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:52.960720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:52.961003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:52.961436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:52.962148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:52.964453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:52.965828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.002588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2800/5663] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:52 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:53.009486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.074752 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1531097391. To ensure interoperability, identifiers should be a valid uuid. Sep 24 21:11:53.074752 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' Sep 24 21:11:53.074752 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:11:53.085968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:53.094470 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2864/5664] 10.222.0.52 () {58 vars in 1317 bytes} [Sat Sep 24 21:11:52 2022] DELETE /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1531097391 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:53.096232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.111891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06496640-31e4-480c-b787-13137e5ea9fe tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.133566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381a30d-a074-4122-84ba-32f9239336c0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.140870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.142220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.142820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.143501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.145378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.148003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.201867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.202606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.202958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.203305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.204153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.206784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.219077 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2801/5665] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:53 2022] POST /identity/v3/projects => generated 359 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:53.220588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.228903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:53.242945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2865/5666] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:53 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:53.244001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.329242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:53.357725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec56d1-017d-4090-b67b-f446bcdf8acb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.367073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cbd0b-52d1-4715-a7a2-c106ab6d1784 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.377107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.377873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.378257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.379872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] PATH_INFO: `/v3/projects/dab1f00857ac4d9e849e6f86a0ae15ce` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.382971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab1f00857ac4d9e849e6f86a0ae15ce)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.387296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.506765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.507938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.507938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.511748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1531097391` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.524650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab1f00857ac4d9e849e6f86a0ae15ce', 'request_id': 'req-d193252d-b94c-4563-8f48-8f819a8f3a59'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:53.534466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=5af2e76deb7c4f54bd423f06ef9f793c, value=tempest-tag-1531097391)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.541262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.544210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab1f00857ac4d9e849e6f86a0ae15ce', 'request_id': 'req-d193252d-b94c-4563-8f48-8f819a8f3a59'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Could not find project tag: tempest-tag-1531097391.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1531097391. Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get Sep 24 21:11:53.567227 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-1531097391. Sep 24 21:11:53.568644 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:53.570562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2866/5667] 10.222.0.52 () {58 vars in 1314 bytes} [Sat Sep 24 21:11:53 2022] GET /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags/tempest-tag-1531097391 => generated 107 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:53.571762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.574427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab1f00857ac4d9e849e6f86a0ae15ce', 'request_id': 'req-d193252d-b94c-4563-8f48-8f819a8f3a59'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:53.599082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] The token cache is being invalidate because project dab1f00857ac4d9e849e6f86a0ae15ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:53.599523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:53.603719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:53.608172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2802/5668] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:53 2022] DELETE /identity/v3/projects/dab1f00857ac4d9e849e6f86a0ae15ce => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:53.612157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.620036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:53.663369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680cf6-02aa-4045-a01e-93d5bda14f4d tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.739933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.740739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.741264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.741647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/67dcfa69abbe4a479944a543a38929c7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.786144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:update_user(user_id=67dcfa69abbe4a479944a543a38929c7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.788794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.866731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193252d-b94c-4563-8f48-8f819a8f3a59 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:53.871973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2867/5669] 10.222.0.52 () {60 vars in 1244 bytes} [Sat Sep 24 21:11:53 2022] PATCH /identity/v3/users/67dcfa69abbe4a479944a543a38929c7 => generated 524 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:53.873143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:53.876859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:53.886835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:53.890102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.890392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.890690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:53.891352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:53.899434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283eac-d3d8-4fd7-8072-5c9c011129e9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:53.906446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:53.907199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:53.956868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2803/5670] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:53.959064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:53.959350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:53.959622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.017028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:54.022243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.037862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2804/5671] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:54.039131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.040547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.067929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1b8ef-2618-473b-825e-b4f6401f5397 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.074039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.075206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.075493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.075771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/67dcfa69abbe4a479944a543a38929c7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.086945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a39c32-cdd6-4c96-99fd-14c130ee9829 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.092179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.092917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.093295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.093715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.095049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=5af2e76deb7c4f54bd423f06ef9f793c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.097443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.103339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:get_user(user_id=67dcfa69abbe4a479944a543a38929c7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.105966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.113204 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2805/5672] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:11:54 2022] GET /identity/v3/users/67dcfa69abbe4a479944a543a38929c7 => generated 390 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:54.114248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.129880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3215-00e4-4722-93b8-6e3ca3754606 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.135974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.136782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.137060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.137333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.137977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.139649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43266bdc-2787-40c8-b085-3997f4c4e65b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.159450 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2806/5673] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:54.161758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.162050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.162325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.176525 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2868/5674] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:11:53 2022] DELETE /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:54.177616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.187251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:54.190084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.193858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e92c18-a018-4ca7-9768-ac57c313e891 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.203373 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2807/5675] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:54.203373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.203683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.204250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.204537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.204817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/projects/07b096200c1a4fb5bb6473bbf88bd452` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.213008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48da1e8-bc1d-45c8-9b1b-be12319270ed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.213857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_project(project_id=07b096200c1a4fb5bb6473bbf88bd452)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.215859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.224286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.225958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.227056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.227575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] PATH_INFO: `/v3/projects/218fe8d2fb984d39945ed03865a2b39e/users/a3a19e32a83f435c89147ae45838c190/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.258485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218fe8d2fb984d39945ed03865a2b39e, user_id=a3a19e32a83f435c89147ae45838c190, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.262634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.291076 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2808/5676] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:54 2022] PUT /identity/v3/projects/218fe8d2fb984d39945ed03865a2b39e/users/a3a19e32a83f435c89147ae45838c190/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:54.292884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.333415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b096200c1a4fb5bb6473bbf88bd452', 'request_id': 'req-fea087d9-271f-44f7-a744-2ff75db61714'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.346464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b096200c1a4fb5bb6473bbf88bd452', 'request_id': 'req-fea087d9-271f-44f7-a744-2ff75db61714'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.361041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b096200c1a4fb5bb6473bbf88bd452', 'request_id': 'req-fea087d9-271f-44f7-a744-2ff75db61714'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.389973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] The token cache is being invalidate because project 07b096200c1a4fb5bb6473bbf88bd452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:54.390418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.392135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.400222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2869/5677] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:54 2022] DELETE /identity/v3/projects/07b096200c1a4fb5bb6473bbf88bd452 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:54.403668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.408303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.415154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c98a1-a451-4589-99e6-a65a061443c7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.508037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.509680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.510131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.510620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.528308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=5af2e76deb7c4f54bd423f06ef9f793c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.531016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb3a7-1f59-4f90-b56e-f7e31721518b tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.550980 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2809/5678] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Sep 24 21:11:54 2022] GET /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c/tags => generated 66 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) Sep 24 21:11:54.553340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.553797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.554170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.558179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:54.568174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea087d9-271f-44f7-a744-2ff75db61714 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.578266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.580345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.580680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.581008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.581731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:54.582779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.585721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.606285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] MFA Rules not processed for user `a3a19e32a83f435c89147ae45838c190`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:54.608587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.619220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2870/5679] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:54.620168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.629199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.662029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:54.752756 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2810/5680] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:54.752756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.755383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:54.795037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cead6-5986-4356-a855-481a378e0115 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.798849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a130-f39b-4939-8016-36f4227b84a0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:54.806004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a09c6192d0a42beabe0f847fef930a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ff4a6b8f1b274032955ba99fee794874', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.806978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.807479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.808071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] PATH_INFO: `/v3/users/67dcfa69abbe4a479944a543a38929c7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.809518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorizing `identity:delete_user(user_id=67dcfa69abbe4a479944a543a38929c7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.813032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.813582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5da4e78ed8ad4a7fab639e0f0475c8ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e2db6b37f824828a46ca070a7b775a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:54.814235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:54.814524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:54.814805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] PATH_INFO: `/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:54.816006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5af2e76deb7c4f54bd423f06ef9f793c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:54.819228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:54.904253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67dcfa69abbe4a479944a543a38929c7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.906151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67dcfa69abbe4a479944a543a38929c7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.921353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67dcfa69abbe4a479944a543a38929c7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.938785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af2e76deb7c4f54bd423f06ef9f793c', 'request_id': 'req-d023e8fd-5e97-404d-8b53-9f3c40b405f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.946126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af2e76deb7c4f54bd423f06ef9f793c', 'request_id': 'req-d023e8fd-5e97-404d-8b53-9f3c40b405f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.952867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67dcfa69abbe4a479944a543a38929c7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.953683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67dcfa69abbe4a479944a543a38929c7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.971605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af2e76deb7c4f54bd423f06ef9f793c', 'request_id': 'req-d023e8fd-5e97-404d-8b53-9f3c40b405f9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:54.977993 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2871/5681] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:54 2022] DELETE /identity/v3/users/67dcfa69abbe4a479944a543a38929c7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:54.980185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:54.988201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.010014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] The token cache is being invalidate because project 5af2e76deb7c4f54bd423f06ef9f793c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:55.011665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.013870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.018788 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2811/5682] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:54 2022] DELETE /identity/v3/projects/5af2e76deb7c4f54bd423f06ef9f793c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:55.020296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.025908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.143833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c113-8cfe-4ee9-a651-974863453500 tempest-UsersV3TestJSON-663732226 tempest-UsersV3TestJSON-663732226-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.159895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.161551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.161842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.162115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.162861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.165143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.203202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:55.227844 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2872/5683] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:54 2022] POST /identity/v3/projects => generated 371 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:55.227844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.229252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.246017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e8fd-5e97-404d-8b53-9f3c40b405f9 tempest-IdentityV3ProjectTagsTest-768751855 tempest-IdentityV3ProjectTagsTest-768751855-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.319715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:55.351198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.352129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.352514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.353669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.355171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.359108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.366653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee90-31cb-40bc-9228-a08a63f0e513 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.380725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.381389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.381666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.382072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] PATH_INFO: `/v3/users/395c85cb5dff4c31a371d2d39fe594a2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.417035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] RBAC: Authorizing `identity:delete_user(user_id=395c85cb5dff4c31a371d2d39fe594a2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.423525 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2812/5684] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/projects => generated 385 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:55.424970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.428449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.441746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.552398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc12f76-afab-4b45-b931-e1bc8c43b3d9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.556916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.558033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.559625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.560432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.560752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.561091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] PATH_INFO: `/v3/users/72343eb12cea4915b2ae7fd342fb6ab1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.573693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72343eb12cea4915b2ae7fd342fb6ab1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.575949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.576470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.599587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.600559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395c85cb5dff4c31a371d2d39fe594a2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.654727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2873/5685] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:55 2022] DELETE /identity/v3/users/395c85cb5dff4c31a371d2d39fe594a2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:55.655853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.657701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.705610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72343eb12cea4915b2ae7fd342fb6ab1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.706303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72343eb12cea4915b2ae7fd342fb6ab1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.718715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72343eb12cea4915b2ae7fd342fb6ab1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.736067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72343eb12cea4915b2ae7fd342fb6ab1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.737295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-633f6c41-553f-4327-8a8f-aa00a7c33393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72343eb12cea4915b2ae7fd342fb6ab1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:55.776114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d097096-9419-48ec-9dd7-07810a58412f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.776969 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2813/5686] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:55 2022] DELETE /identity/v3/users/72343eb12cea4915b2ae7fd342fb6ab1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:55.785617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.785617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.785617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.787381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:55.821930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] MFA Rules not processed for user `b2980f6962204dd6a16c5f7d1ed2e127`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:55.823095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.845452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.846166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.846652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.846993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.847636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.849780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.875123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:55.878284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2874/5687] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:55.880097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.882070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.897596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e44a-ffac-4dcc-bc51-bb2b539683e8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.904552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.905442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.905724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.906552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.906675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.908689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1893-c33d-4b11-979e-ff8c64ddff97 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.919391 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2814/5688] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:55.920295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:55.924624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.942384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2875/5689] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/users => generated 299 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:55.944124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579e773-7051-45da-bfc1-5f307b09fddc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.944510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579e773-7051-45da-bfc1-5f307b09fddc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.944889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579e773-7051-45da-bfc1-5f307b09fddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.946259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36940271-d73f-413c-8f90-17b1e9b650da None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:55.953349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:55.953546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.953881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.954156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] PATH_INFO: `/v3/users/a22830d73cb043b8b9c2e070afa7af55` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:55.955991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22830d73cb043b8b9c2e070afa7af55)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:55.959194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:55.971070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4579e773-7051-45da-bfc1-5f307b09fddc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:55.972540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4579e773-7051-45da-bfc1-5f307b09fddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:55.991460 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2876/5690] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:55.994734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:55.995133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:55.995751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.023949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:56.025335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.036139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22830d73cb043b8b9c2e070afa7af55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.037927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22830d73cb043b8b9c2e070afa7af55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.040104 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2877/5691] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:56.040781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.042601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.055351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22830d73cb043b8b9c2e070afa7af55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.064038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e15e3a-88b6-4f6c-b42a-4955ba86235b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.067867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22830d73cb043b8b9c2e070afa7af55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.069157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22830d73cb043b8b9c2e070afa7af55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.071887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.072798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.073119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.073487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.074232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.076137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.088047 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2878/5692] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:56.089104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.091320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.104410 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2815/5693] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:55 2022] DELETE /identity/v3/users/a22830d73cb043b8b9c2e070afa7af55 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:56.105589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.107039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.110637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2272ce-a50a-46ea-b47b-ffc5e95428cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.192613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.194682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.201012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.201734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.203444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.205845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.235179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2879/5694] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:56.236110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.248843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b5492-2807-460e-9f01-0d7860a09bb1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.251585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70a0f1-676c-4ad3-9d91-d50eee3897c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.255841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.256305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.256838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.257465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.257465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.257721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.258318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.259707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.261199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.263536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.278019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.280266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.294578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2880/5695] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:56.294903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.295925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.296683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.298266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.309064 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2816/5696] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:56.310121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.317259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193958a-64cc-4950-a041-28b50c8ab07f service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.323128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.324014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.324475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566cd22-d77a-4d40-9b20-9e8d2e34525c service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.324742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.325338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.326221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.328864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.331105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.332613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.333228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.333660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] PATH_INFO: `/v3/projects/271c84f9d87645c9b9f0570939ae73c9/users/ef42892154f442649c8814026114a00e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.351993 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2881/5697] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:56.355842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.366162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271c84f9d87645c9b9f0570939ae73c9, user_id=ef42892154f442649c8814026114a00e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.367966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983df1-47f5-4511-945b-ea817e968d3d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.369661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.373769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.374452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.374757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.375018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] PATH_INFO: `/v3/projects/d6f9abd4cfb1462da57590aa5f311707/users/25437c63ca5444e584e1ae81a40c43e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.399300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2817/5698] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:56 2022] PUT /identity/v3/projects/271c84f9d87645c9b9f0570939ae73c9/users/ef42892154f442649c8814026114a00e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:56.400907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.406724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f9abd4cfb1462da57590aa5f311707, user_id=25437c63ca5444e584e1ae81a40c43e4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.411957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514f0b6-3f4f-47aa-90fe-3a7e2682c9e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.436259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:56.444208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2882/5699] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:56 2022] PUT /identity/v3/projects/d6f9abd4cfb1462da57590aa5f311707/users/25437c63ca5444e584e1ae81a40c43e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:11:56.450252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.450909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.452264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.478231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:56.504979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] MFA Rules not processed for user `25437c63ca5444e584e1ae81a40c43e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:56.506638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.519372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bccb3-6d1f-49e8-bf76-640afec301a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.624750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.625376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.625710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.625992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.626723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.628425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.635461 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2883/5700] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:56.635461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.635461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.643113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2818/5701] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:56.647990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.652650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f7d41-de89-4a87-bedd-ccb46708adf9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.661217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.661217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.661217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.661812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] PATH_INFO: `/v3/projects/b170a25f4e4d44ba88fffcdc51d3fbbe` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.663444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b170a25f4e4d44ba88fffcdc51d3fbbe)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.666308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.670709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dace95-c7f2-4192-a598-324627e7050c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.677970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.679206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.679732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.680223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.681138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.682968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.710564 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2819/5702] 10.222.0.52 () {60 vars in 1299 bytes} [Sat Sep 24 21:11:56 2022] POST /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:56.711382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.727680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553bcaf-2944-4fb5-bcaa-d57d9d8c8ddd tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.743254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b170a25f4e4d44ba88fffcdc51d3fbbe', 'request_id': 'req-59006aed-75e0-461c-b59c-c5f3df72c316'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.749659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b170a25f4e4d44ba88fffcdc51d3fbbe', 'request_id': 'req-59006aed-75e0-461c-b59c-c5f3df72c316'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.756037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b170a25f4e4d44ba88fffcdc51d3fbbe', 'request_id': 'req-59006aed-75e0-461c-b59c-c5f3df72c316'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.771360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] The token cache is being invalidate because project b170a25f4e4d44ba88fffcdc51d3fbbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:56.771668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.772396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:56.778587 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2884/5703] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:56 2022] DELETE /identity/v3/projects/b170a25f4e4d44ba88fffcdc51d3fbbe => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:56.783246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:56.785737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.810420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:56.811350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.811831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.812109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] PATH_INFO: `/v3/projects/271c84f9d87645c9b9f0570939ae73c9/users/ef42892154f442649c8814026114a00e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.815159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=271c84f9d87645c9b9f0570939ae73c9, user_id=ef42892154f442649c8814026114a00e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:56.817901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4ab7c-11de-4620-b881-a7c88f659858 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:56.840268 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2820/5704] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:56 2022] PUT /identity/v3/projects/271c84f9d87645c9b9f0570939ae73c9/users/ef42892154f442649c8814026114a00e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:56.840268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b231e-cf0f-4820-b163-c6555f4180ab None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.840268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b231e-cf0f-4820-b163-c6555f4180ab None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.840268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b231e-cf0f-4820-b163-c6555f4180ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:56.872906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-389b231e-cf0f-4820-b163-c6555f4180ab None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:56.874458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-389b231e-cf0f-4820-b163-c6555f4180ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:56.900696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59006aed-75e0-461c-b59c-c5f3df72c316 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:56.946821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:56.959522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:56.995207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2821/5705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:56.996832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:56.997127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:56.997918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.038846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] MFA Rules not processed for user `ef42892154f442649c8814026114a00e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:57.040023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.047636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.049012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.051024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.051714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/be2c17b5488f45c1a44f1598ba91f9e3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.057865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=be2c17b5488f45c1a44f1598ba91f9e3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.059867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90b4f8-9d54-4fb3-88d2-6df43b2287ab tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.103689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2885/5706] 10.222.0.52 () {58 vars in 1380 bytes} [Sat Sep 24 21:11:56 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/be2c17b5488f45c1a44f1598ba91f9e3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.108281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.108807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.110610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.179419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:57.180676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.185081 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2822/5707] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:57.186816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.199847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab5300-2ae8-4baf-bef8-435aa620c2ba None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.200330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2886/5708] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:57.201283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.208206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.210127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.210883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.211126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.211412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.212242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.213739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.222372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ac45d-94c8-4517-ad84-e676a5dad7c3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.235159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.237313 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2823/5709] 10.222.0.52 () {60 vars in 1299 bytes} [Sat Sep 24 21:11:57 2022] POST /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:57.238085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.238277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.239040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.239544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] PATH_INFO: `/v3/projects/21cf53366d5e4236800a13b32efcf11b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.240427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.242710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cf53366d5e4236800a13b32efcf11b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.252101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.259653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20802766-ce07-4e17-ae65-9cf6461858ff tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.265542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.266193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.266470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.266978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] PATH_INFO: `/v3/projects/2320b653932749b2bba237f24235cbf1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.268168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2320b653932749b2bba237f24235cbf1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.271868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.340714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2320b653932749b2bba237f24235cbf1', 'request_id': 'req-1fe09c35-82d6-499c-af37-4bfd92c00e55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.346845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2320b653932749b2bba237f24235cbf1', 'request_id': 'req-1fe09c35-82d6-499c-af37-4bfd92c00e55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.354855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2320b653932749b2bba237f24235cbf1', 'request_id': 'req-1fe09c35-82d6-499c-af37-4bfd92c00e55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.363976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cf53366d5e4236800a13b32efcf11b', 'request_id': 'req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.370402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] The token cache is being invalidate because project 2320b653932749b2bba237f24235cbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:57.370888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.371675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cf53366d5e4236800a13b32efcf11b', 'request_id': 'req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.371891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.373740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2824/5710] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:57 2022] DELETE /identity/v3/projects/2320b653932749b2bba237f24235cbf1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.374574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.375944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.381334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cf53366d5e4236800a13b32efcf11b', 'request_id': 'req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.396455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] The token cache is being invalidate because project 21cf53366d5e4236800a13b32efcf11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:57.396854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.397569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.399520 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2887/5711] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:57 2022] DELETE /identity/v3/projects/21cf53366d5e4236800a13b32efcf11b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.400281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.401577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.439957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:57.473594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09c35-82d6-499c-af37-4bfd92c00e55 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.476161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fb2e-e46e-4bcf-9e85-e45502fc3f0b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.481461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.482073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.482356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.482669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.483322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:11:57.483639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.485227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.511803 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2888/5712] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:57.512811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.516645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.559180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.559800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.560076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.560512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.563966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.648606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:57.692614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38769d5-ec80-4726-8a9d-211b2716a08d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.702011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.703259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.703522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.703946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.704238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.704524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e9a93dc68f1449e6964022523b316368` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.715330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=e9a93dc68f1449e6964022523b316368)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.716467 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2825/5713] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:57.717268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.719114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.728007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.740147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065ca7-6e16-463d-a188-b2bdd708e209 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.755311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.756027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.756385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.757435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] PATH_INFO: `/v3/users/5da4e78ed8ad4a7fab639e0f0475c8ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.758307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da4e78ed8ad4a7fab639e0f0475c8ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.762256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.786863 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2889/5714] 10.222.0.52 () {58 vars in 1380 bytes} [Sat Sep 24 21:11:57 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e9a93dc68f1449e6964022523b316368 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.788297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.790115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.808766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d2cfb-abfc-4c4c-954d-e08c63d012a8 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:57.815770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:57.817857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:57.818384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:57.818765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] PATH_INFO: `/v3/users/5a09c6192d0a42beabe0f847fef930a7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:57.823574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a09c6192d0a42beabe0f847fef930a7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:57.826388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:57.864787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da4e78ed8ad4a7fab639e0f0475c8ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.865797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da4e78ed8ad4a7fab639e0f0475c8ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.881892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da4e78ed8ad4a7fab639e0f0475c8ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.896754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da4e78ed8ad4a7fab639e0f0475c8ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.898192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da4e78ed8ad4a7fab639e0f0475c8ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.929598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a09c6192d0a42beabe0f847fef930a7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.931425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a09c6192d0a42beabe0f847fef930a7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.932945 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2826/5715] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:57 2022] DELETE /identity/v3/users/5da4e78ed8ad4a7fab639e0f0475c8ad => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.933869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.935588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:57.946074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a09c6192d0a42beabe0f847fef930a7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.953678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:57.969448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a09c6192d0a42beabe0f847fef930a7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.970651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a09c6192d0a42beabe0f847fef930a7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:57.989227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2890/5716] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:57 2022] DELETE /identity/v3/users/5a09c6192d0a42beabe0f847fef930a7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:57.990155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:57.991631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.064809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f746d2-2746-4a5c-b8bc-9c01a5ed207e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.082952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4f14-effd-40f5-942c-09b1b9312b6a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.088468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.089554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.090312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.090609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e9a93dc68f1449e6964022523b316368` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Could not find credential: 4f555bdc6ea4748d9366a1965fac0cee0683940a4591c73f7b8b015449d392f5.: keystone.exception.CredentialNotFound: Could not find credential: 4f555bdc6ea4748d9366a1965fac0cee0683940a4591c73f7b8b015449d392f5. Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get Sep 24 21:11:58.138163 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:11:58.139302 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 4f555bdc6ea4748d9366a1965fac0cee0683940a4591c73f7b8b015449d392f5. Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:11:58.140357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2891/5717] 10.222.0.52 () {58 vars in 1377 bytes} [Sat Sep 24 21:11:57 2022] GET /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e9a93dc68f1449e6964022523b316368 => generated 148 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:11:58.142978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:58.144851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.164445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.165879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.166328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.166597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.167825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.170200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.216632 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2827/5718] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:11:57 2022] POST /identity/v3/projects => generated 401 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:58.217824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:58.220702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.228720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:58.264999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbccbd-25c8-4c4c-8f5f-a1b8ab0d47e9 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.382977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fac01b-1266-4662-bf2c-bf520fddc9be admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.399440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.400188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.400482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.400899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] PATH_INFO: `/v3/users/37fee5c58bad45aca56e2bb3b7a3af85` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.403608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fee5c58bad45aca56e2bb3b7a3af85)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.407049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.427051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.427717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.427998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.428273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] PATH_INFO: `/v3/users/a3a19e32a83f435c89147ae45838c190` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.430708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a19e32a83f435c89147ae45838c190)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.451416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.501871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fee5c58bad45aca56e2bb3b7a3af85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.502791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fee5c58bad45aca56e2bb3b7a3af85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.505529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:58.524408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fee5c58bad45aca56e2bb3b7a3af85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.558188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fee5c58bad45aca56e2bb3b7a3af85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.562759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fee5c58bad45aca56e2bb3b7a3af85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.606843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a19e32a83f435c89147ae45838c190'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.607875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a19e32a83f435c89147ae45838c190'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.614664 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2828/5719] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:58 2022] DELETE /identity/v3/users/37fee5c58bad45aca56e2bb3b7a3af85 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:58.615070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:58.622917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.623836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a19e32a83f435c89147ae45838c190'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.692877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a19e32a83f435c89147ae45838c190'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.695329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a19e32a83f435c89147ae45838c190'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:58.721725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2892/5720] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:58 2022] DELETE /identity/v3/users/a3a19e32a83f435c89147ae45838c190 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:58.723303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:58.724739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.817790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3556f4-9e72-4578-ac02-04e778a0bdea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.819667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e022b3-86ca-4791-bc36-86c9237bc612 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.832242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.832959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.833230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.833517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.834197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.840477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.899855 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2893/5721] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:11:58 2022] POST /identity/v3/users => generated 301 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:58.899855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:58.902954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.905263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.907889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.908190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.908467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.909121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.917933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9138-997e-4549-b89f-e668b4dd7a0b tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.931671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:58.936682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb37-55ea-4072-9a09-16b4c1a8622a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:58.955274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:58.956061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.956449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.956960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2829/5722] 10.222.0.52 () {60 vars in 1299 bytes} [Sat Sep 24 21:11:58 2022] POST /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 367 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:58.957623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] PATH_INFO: `/v3/projects/ff4a6b8f1b274032955ba99fee794874` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.958083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:58.958368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:58.958683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:58.959006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4a6b8f1b274032955ba99fee794874)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:58.961154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:58.982161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:58.983410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:58.998585 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2830/5723] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:58.998585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.001801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.023448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608ecf3-9482-4790-87bf-36458bffda2e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.030842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4a6b8f1b274032955ba99fee794874', 'request_id': 'req-afac151c-a1fd-433d-b072-c1d42ce2b76b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.041578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4a6b8f1b274032955ba99fee794874', 'request_id': 'req-afac151c-a1fd-433d-b072-c1d42ce2b76b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.052185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4a6b8f1b274032955ba99fee794874', 'request_id': 'req-afac151c-a1fd-433d-b072-c1d42ce2b76b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.074031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] The token cache is being invalidate because project ff4a6b8f1b274032955ba99fee794874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:59.074393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.076671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.079293 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2894/5724] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:58 2022] DELETE /identity/v3/projects/ff4a6b8f1b274032955ba99fee794874 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:59.080089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.087309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.111052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.112304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.112467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.112771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] PATH_INFO: `/v3/projects/3e2db6b37f824828a46ca070a7b775a7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.115156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2db6b37f824828a46ca070a7b775a7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.117596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.131390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:59.173404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac151c-a1fd-433d-b072-c1d42ce2b76b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.177836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2db6b37f824828a46ca070a7b775a7', 'request_id': 'req-5337c7f8-a3f1-477e-982c-5b67adcbdc17'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.184833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2db6b37f824828a46ca070a7b775a7', 'request_id': 'req-5337c7f8-a3f1-477e-982c-5b67adcbdc17'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.194700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2db6b37f824828a46ca070a7b775a7', 'request_id': 'req-5337c7f8-a3f1-477e-982c-5b67adcbdc17'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.211114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] The token cache is being invalidate because project 3e2db6b37f824828a46ca070a7b775a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:59.211726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.212239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.220245 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2831/5725] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:58 2022] DELETE /identity/v3/projects/3e2db6b37f824828a46ca070a7b775a7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:59.221612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.223623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.255671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.256812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.257163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.257645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] PATH_INFO: `/v3/projects/77274d03cfb74d738b3b75c92e4cd641` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.259045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77274d03cfb74d738b3b75c92e4cd641)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.261775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.296304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337c7f8-a3f1-477e-982c-5b67adcbdc17 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.301855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.302429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.302858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.303140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.303786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.305406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.314104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:59.320543 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2832/5726] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:59.320543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.322287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.327065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77274d03cfb74d738b3b75c92e4cd641', 'request_id': 'req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.334999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77274d03cfb74d738b3b75c92e4cd641', 'request_id': 'req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.342166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77274d03cfb74d738b3b75c92e4cd641', 'request_id': 'req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.357932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] The token cache is being invalidate because project 77274d03cfb74d738b3b75c92e4cd641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:11:59.358222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.358962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.360887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2895/5727] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:11:59 2022] DELETE /identity/v3/projects/77274d03cfb74d738b3b75c92e4cd641 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:59.361579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.363531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.399655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39027984-3f6d-47d4-9d77-1397015c7299 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.435969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8e2de-4388-442c-ba90-aeb6e4a533d0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.440546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.441200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.441485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.441757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.443761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.460753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:59.473448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.474102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.474388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.474845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.475485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.477644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e90d90-2535-4b5a-b653-9c3d819c697c tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.514529 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2833/5728] 10.222.0.52 () {60 vars in 1299 bytes} [Sat Sep 24 21:11:59 2022] POST /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:11:59.514529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720499-d454-4bef-a7e5-2b2215be654f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.514529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720499-d454-4bef-a7e5-2b2215be654f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.514529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c720499-d454-4bef-a7e5-2b2215be654f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.525841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6c720499-d454-4bef-a7e5-2b2215be654f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:59.527546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6c720499-d454-4bef-a7e5-2b2215be654f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.530482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.531969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.539230 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2834/5729] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:59.540539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.540878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.541287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.543151 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2896/5730] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:11:59.543831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.545161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.559566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc90c-168f-4a3a-a38c-da62e7765e02 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.563416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:59.565081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.565340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.574199 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2835/5731] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:59.575547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.577432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.578325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.579834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.580220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.580901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.583546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.598879 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2897/5732] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:11:59.598879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.599306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c94726-fca3-482d-9f7c-449e03235f9d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.601554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.614554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.615191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.615469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.615742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] PATH_INFO: `/v3/users/b2980f6962204dd6a16c5f7d1ed2e127` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.616941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2980f6962204dd6a16c5f7d1ed2e127)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.619627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.633563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5da73c-b0be-46ca-ab20-252863ecceca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.638329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.639131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.639417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.639707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.640269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.641876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8b671-f390-44f4-ac62-d000c2cec344 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.663320 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2898/5733] 10.222.0.52 () {58 vars in 1278 bytes} [Sat Sep 24 21:11:59 2022] GET /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 865 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:11:59.665685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccaafb-559a-486d-b813-8d54e0dcadfc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.665978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccaafb-559a-486d-b813-8d54e0dcadfc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.666444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccaafb-559a-486d-b813-8d54e0dcadfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.697828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2980f6962204dd6a16c5f7d1ed2e127'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.698553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2980f6962204dd6a16c5f7d1ed2e127'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.710913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2980f6962204dd6a16c5f7d1ed2e127'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.721767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2980f6962204dd6a16c5f7d1ed2e127'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.723206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2980f6962204dd6a16c5f7d1ed2e127'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:11:59.723783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-71ccaafb-559a-486d-b813-8d54e0dcadfc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:59.725859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-71ccaafb-559a-486d-b813-8d54e0dcadfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.741931 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2899/5734] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:59.743322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.743611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.743882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.751973 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2836/5735] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:11:59 2022] DELETE /identity/v3/users/b2980f6962204dd6a16c5f7d1ed2e127 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:59.753226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.756851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.777615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:11:59.779330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.840707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:11:59.870282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c75b7-5d22-447c-844b-11a7fc88d80d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.870959 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2900/5736] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:11:59.871853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.873363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:11:59.880244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:11:59.881540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:11:59.881999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:11:59.882295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] PATH_INFO: `/v3/projects/55dac84d9bd0493286f8dbdab75b5904/users/148377c3b25b4fb7bcef8159b13499f1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:11:59.906050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=55dac84d9bd0493286f8dbdab75b5904, user_id=148377c3b25b4fb7bcef8159b13499f1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:11:59.908835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:11:59.934611 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2837/5737] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:11:59 2022] PUT /identity/v3/projects/55dac84d9bd0493286f8dbdab75b5904/users/148377c3b25b4fb7bcef8159b13499f1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:11:59.935590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:11:59.972588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcfad3-7b78-4a9b-849e-6703b2cfb77f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:11:59.996222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:00.062860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.063690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.064268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.064718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e28ae9da2afe40329de04aa192cde893` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.067678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=e28ae9da2afe40329de04aa192cde893)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.069252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.093124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2901/5738] 10.222.0.52 () {58 vars in 1380 bytes} [Sat Sep 24 21:11:59 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/e28ae9da2afe40329de04aa192cde893 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:00.093361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a6ca-3a12-4b49-9117-1a1d4f85d2af admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.094776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.098761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.099149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.099422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.099692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.100343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:00.100640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.102239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82805f00-f223-4e3a-a4fa-ad7d4418c168 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.109724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7b7c1-282e-4e06-bd87-b4695d847f50 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.114698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.115338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.115708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.115878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.116522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:00.116821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.121614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.121765 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2838/5739] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:00.121765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.121765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.121765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.136600 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2902/5740] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:00.136600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.138475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.157344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] MFA Rules not processed for user `148377c3b25b4fb7bcef8159b13499f1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:00.157539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f37a-e019-4cb4-9237-067fa348771d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.158976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.163185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.163984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.164336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.164802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] PATH_INFO: `/v3/projects/26ca384451ad438eb7d7e074fd11a122` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.166006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ca384451ad438eb7d7e074fd11a122)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.168053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.235660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ca384451ad438eb7d7e074fd11a122', 'request_id': 'req-a0e03d71-de0b-433d-983b-58228f465baa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.247268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ca384451ad438eb7d7e074fd11a122', 'request_id': 'req-a0e03d71-de0b-433d-983b-58228f465baa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.264179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ca384451ad438eb7d7e074fd11a122', 'request_id': 'req-a0e03d71-de0b-433d-983b-58228f465baa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.292230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] The token cache is being invalidate because project 26ca384451ad438eb7d7e074fd11a122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:00.292615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.294417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2839/5741] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:00.295549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.297756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.300243 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2903/5742] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:00 2022] DELETE /identity/v3/projects/26ca384451ad438eb7d7e074fd11a122 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:00.301801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.309718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.323543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180ebec-c350-4e01-9f41-86657a62703c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.327709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:00.412913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e03d71-de0b-433d-983b-58228f465baa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.418228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.419253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.419253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.419490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.421841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.421841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367a27f-0597-431d-8ace-211e49e8c3bc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.436108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.437367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.437925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.438270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/2f643eecb03a4597957559410693c280` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.443302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=2f643eecb03a4597957559410693c280)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.445746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.469930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2840/5743] 10.222.0.52 () {58 vars in 1380 bytes} [Sat Sep 24 21:12:00 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/2f643eecb03a4597957559410693c280 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:00.472889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.482725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2904/5744] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:00.485748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.485957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.486220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.497134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.523993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6559b-8ef4-428f-88c1-1da7ec4fe003 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.529442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:00.533324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.615167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:00.633141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.633822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.634071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.634518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.635240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.639434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.658600 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2905/5745] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:00.658600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.666356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.682003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773369d-d989-42bd-b64f-c405bec9b738 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.683693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2841/5746] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:00.683693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.691088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.778214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.779005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.781027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.781448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] PATH_INFO: `/v3/users/148377c3b25b4fb7bcef8159b13499f1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.783694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148377c3b25b4fb7bcef8159b13499f1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.785666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1a344-a7e8-478a-ab87-6839e3bec67f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.789727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.795623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.796781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.797076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.797514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] PATH_INFO: `/v3/projects/218fe8d2fb984d39945ed03865a2b39e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.799530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] RBAC: Authorizing `identity:delete_project(project_id=218fe8d2fb984d39945ed03865a2b39e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.802469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:00.842923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:00.859168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148377c3b25b4fb7bcef8159b13499f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.860113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148377c3b25b4fb7bcef8159b13499f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.870877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148377c3b25b4fb7bcef8159b13499f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.874685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218fe8d2fb984d39945ed03865a2b39e', 'request_id': 'req-28094a84-f5fc-40e8-bd8d-4d583375c80b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.881403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218fe8d2fb984d39945ed03865a2b39e', 'request_id': 'req-28094a84-f5fc-40e8-bd8d-4d583375c80b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.894873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148377c3b25b4fb7bcef8159b13499f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.895611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148377c3b25b4fb7bcef8159b13499f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.898690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218fe8d2fb984d39945ed03865a2b39e', 'request_id': 'req-28094a84-f5fc-40e8-bd8d-4d583375c80b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.910541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2906/5747] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:00 2022] DELETE /identity/v3/users/148377c3b25b4fb7bcef8159b13499f1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:00.910541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.912225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.921782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] The token cache is being invalidate because project 218fe8d2fb984d39945ed03865a2b39e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:00.922203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.923972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:00.926972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2842/5748] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:00 2022] DELETE /identity/v3/projects/218fe8d2fb984d39945ed03865a2b39e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:00.931140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:00.932698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:00.985256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d0af9-832f-4aca-ac99-ac89184cfb14 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:00.991363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:00.992084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:00.992377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:00.992710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:00.993343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:00.995047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671798-2e53-4515-8482-89782ad48aee tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.016609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094a84-f5fc-40e8-bd8d-4d583375c80b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.020352 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2907/5749] 10.222.0.52 () {60 vars in 1299 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2 => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:01.020352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.020352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.020802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.025924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:01.047916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:01.049570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.095339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.096070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.096426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.096771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.098491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.101019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ae83b-88fb-4ac2-9635-f0035397172e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.119776 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2908/5750] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:01.120332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.121855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.129760 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2843/5751] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:00 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:01.129760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.129760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.129760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.134808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865a3f6-c193-4d39-a0f4-9b9f505f1d49 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.140022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.140624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.140913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.141165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.141792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.143734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.152698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:01.153983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.167637 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2844/5752] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:01.167637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.168327 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2909/5753] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:01 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:01.169099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.170758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.172229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.184189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed4425-0318-49f7-a065-6b214e6bdb23 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.188392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.189037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.189317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.189590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/bf1db4446b6a48b0a1340c1237fddd02` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.190444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a844469-1c37-48bc-91a3-5005ed5a7e84 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.191696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=bf1db4446b6a48b0a1340c1237fddd02)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.193180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.196324 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2910/5754] 10.222.0.52 () {58 vars in 1377 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/bf1db4446b6a48b0a1340c1237fddd02 => generated 367 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:01.197202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.201906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.202791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.203133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.203488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] PATH_INFO: `/v3/users/ef42892154f442649c8814026114a00e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.204750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef42892154f442649c8814026114a00e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.206789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.213028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334673de-3f5e-4af4-81de-9dae9cc8536e tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.218357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.219083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.219373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.219862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.220648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:01.220958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.223311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.240404 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2911/5755] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:01.241378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.252492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d214bda-e900-43f6-8800-3e021770e357 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.257540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.258142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.258425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.258824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.259464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.261058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.268730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef42892154f442649c8814026114a00e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.269966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef42892154f442649c8814026114a00e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.272910 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2912/5756] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:01.274530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.276745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.280041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef42892154f442649c8814026114a00e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.287790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:01.289315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef42892154f442649c8814026114a00e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.290155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef42892154f442649c8814026114a00e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.292274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a86d5-fb82-4286-a469-8c255349e7c7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.299662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.300286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.301155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.301547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.302614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.304243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.318684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2845/5757] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:01 2022] DELETE /identity/v3/users/ef42892154f442649c8814026114a00e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:01.319599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.325283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.341048 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2913/5758] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:01.342842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.345087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.361562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55f709-3cfa-4582-8164-ebb1ca516799 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.415194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6c67-115a-425d-992e-926ff3ce2ee8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.419505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.420062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.420342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.420615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.423528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.438223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25437c63ca5444e584e1ae81a40c43e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6f9abd4cfb1462da57590aa5f311707', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.439005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.439489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.439825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/bf1db4446b6a48b0a1340c1237fddd02` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.442582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=25437c63ca5444e584e1ae81a40c43e4, credential_id=bf1db4446b6a48b0a1340c1237fddd02)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.444380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.445556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:01.473440 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2914/5759] 10.222.0.52 () {58 vars in 1380 bytes} [Sat Sep 24 21:12:01 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4/credentials/OS-EC2/bf1db4446b6a48b0a1340c1237fddd02 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:01.473440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.475202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.512590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.514077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.527138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2846/5760] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:01.528013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.529510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.548106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce839159-e4c0-4a6f-a5e7-0b9e23b83648 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.561533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.562783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.563066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.563342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.565174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.568421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.595241 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2847/5761] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:01.596773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.609315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078708c-062c-479b-a422-2b209814a908 tempest-EC2CredentialsTest-736485486 tempest-EC2CredentialsTest-736485486-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.623401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.624056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.624342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.624633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.625267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.641676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b609c3-0cf5-4a68-866f-ed846d21e173 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.642854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.658125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.661814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.661814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.661814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.661814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.675429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.695414 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2848/5762] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:01.699677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.702191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.717891 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2915/5763] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:01 2022] POST /identity/v3/projects => generated 377 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:01.718871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.742083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e188-7ccb-465b-9204-23becb0d7c4c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.814311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.815527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.816305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.816646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] PATH_INFO: `/v3/users/25437c63ca5444e584e1ae81a40c43e4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.818050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25437c63ca5444e584e1ae81a40c43e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.820151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.873748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf3ba-1674-45d5-a062-82c4e9f1a343 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:01.881067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:01.882505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:01.884433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:01.884777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] PATH_INFO: `/v3/projects/1d08d749ed5c432ca9776526cd9aebd7/users/5d572a1f774345018fd0fbaf161a6fdf/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:01.911135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d08d749ed5c432ca9776526cd9aebd7, user_id=5d572a1f774345018fd0fbaf161a6fdf, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:01.915855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:01.930084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25437c63ca5444e584e1ae81a40c43e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.930261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:01.935793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25437c63ca5444e584e1ae81a40c43e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.949213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2916/5764] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:01 2022] PUT /identity/v3/projects/1d08d749ed5c432ca9776526cd9aebd7/users/5d572a1f774345018fd0fbaf161a6fdf/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:01.950753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:01.953499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:01.964199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25437c63ca5444e584e1ae81a40c43e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.985469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25437c63ca5444e584e1ae81a40c43e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:01.986246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25437c63ca5444e584e1ae81a40c43e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.014266 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2849/5765] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:01 2022] DELETE /identity/v3/users/25437c63ca5444e584e1ae81a40c43e4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:02.015449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.016729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.054819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:02.080531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c435c79-88de-4eb8-9b0b-6fdd08783340 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.151885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f02819-b8a0-40e1-a46a-850345417aa5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.157363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.158076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.158455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.158851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.159577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.163025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e431ee7-0fb5-426e-ae6b-4fb15bc4b4ec admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.180036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.180767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.181397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.181720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] PATH_INFO: `/v3/projects/b18f80a424de46a089b6ba60df77cf48/users/9caa557423fd43069bec6eccce15169b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.189552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2850/5766] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:02.191198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.191780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.192274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.206436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18f80a424de46a089b6ba60df77cf48, user_id=9caa557423fd43069bec6eccce15169b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.209956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cb413-bf92-4eb9-9175-92c8e01a056c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.233525 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2917/5767] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:01 2022] PUT /identity/v3/projects/b18f80a424de46a089b6ba60df77cf48/users/9caa557423fd43069bec6eccce15169b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:02.235862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.236357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.236792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.238226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] MFA Rules not processed for user `5d572a1f774345018fd0fbaf161a6fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.239655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.266355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.267834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.345820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2851/5768] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:02.347267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.349600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.359749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2918/5769] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:02.360730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.373816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aec3a9-9e49-4998-8856-abaf13614fc6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.375787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c095-09c7-49ba-a7d6-ee7a4d90f9f3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.379746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.380708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.380999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.381289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] PATH_INFO: `/v3/projects/271c84f9d87645c9b9f0570939ae73c9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.382451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] RBAC: Authorizing `identity:delete_project(project_id=271c84f9d87645c9b9f0570939ae73c9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.382750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.383437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.383788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.384140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.384846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.385054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.386617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.399432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2919/5770] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:02.400525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.402792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.417758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fff2a-8d03-4ada-aac3-1d392681c1ef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.425329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.426911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.427188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.427461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] PATH_INFO: `/v3/projects/55dac84d9bd0493286f8dbdab75b5904` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.428634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55dac84d9bd0493286f8dbdab75b5904)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.430644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.444884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:02.459930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271c84f9d87645c9b9f0570939ae73c9', 'request_id': 'req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.466371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271c84f9d87645c9b9f0570939ae73c9', 'request_id': 'req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.475991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271c84f9d87645c9b9f0570939ae73c9', 'request_id': 'req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.489349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55dac84d9bd0493286f8dbdab75b5904', 'request_id': 'req-6166b753-384e-46a1-9021-c694c4d7a025'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.490865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] The token cache is being invalidate because project 271c84f9d87645c9b9f0570939ae73c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:02.491218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.491956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30cb93f8-b20e-4a99-a7a4-c5bc5fea695f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.493732 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2852/5771] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:02 2022] DELETE /identity/v3/projects/271c84f9d87645c9b9f0570939ae73c9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:02.495807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.496168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.496828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.501330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55dac84d9bd0493286f8dbdab75b5904', 'request_id': 'req-6166b753-384e-46a1-9021-c694c4d7a025'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.509539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55dac84d9bd0493286f8dbdab75b5904', 'request_id': 'req-6166b753-384e-46a1-9021-c694c4d7a025'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.527599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] The token cache is being invalidate because project 55dac84d9bd0493286f8dbdab75b5904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:02.527784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:02.528394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.528992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:02.534559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2920/5772] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:02 2022] DELETE /identity/v3/projects/55dac84d9bd0493286f8dbdab75b5904 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:02.534559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.534559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.536471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] MFA Rules not processed for user `9caa557423fd43069bec6eccce15169b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.539025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.617582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166b753-384e-46a1-9021-c694c4d7a025 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.627404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.628228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.628701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.636315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.636530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2853/5773] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:02.636626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.636718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.637994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.638103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.676757 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2921/5774] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:02.677768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.679990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.711657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0e1d-84ba-45ff-bdd4-78f894f34c6c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.763687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de16fe7-4ef4-4a77-bfa4-378ddaff1968 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.773267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.775244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.775624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.775896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.778193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.780152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc0122-f243-4b58-bbb8-45c20396c2eb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.796240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2922/5775] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:02.796476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.796589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeaaf-b83a-41d2-8419-31897e905b08 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.796589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeaaf-b83a-41d2-8419-31897e905b08 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.796776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.796855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efeaaf-b83a-41d2-8419-31897e905b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.796932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.796932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.797909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.818410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.820257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.820433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84efeaaf-b83a-41d2-8419-31897e905b08 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.821667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84efeaaf-b83a-41d2-8419-31897e905b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.838605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2923/5776] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:02.838605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.838605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.838605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.848823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2854/5777] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:02.850065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.851666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.861638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.864054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:02.866568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fbd2e-0fd6-491d-95a5-6686a18e630a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.878656 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2924/5778] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:02.880000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.887610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.888235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.888518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.888794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.889436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.891535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.903505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178fb6-3666-4b19-aec9-c52da798c241 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:02.911411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:02.912469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.912755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.913031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.913675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:02.918882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972b2bc-c783-40c4-a09a-d98a6bbb263d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:02.929312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:02.942240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2855/5779] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:02.942240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:02.951593 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2925/5780] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/users => generated 291 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:02.952917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4568d-3d32-41cf-b82d-48dcef9375ba None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:02.953143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4568d-3d32-41cf-b82d-48dcef9375ba None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:02.953429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c4568d-3d32-41cf-b82d-48dcef9375ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:02.986714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b3c4568d-3d32-41cf-b82d-48dcef9375ba None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:02.988475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3c4568d-3d32-41cf-b82d-48dcef9375ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.008134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:03.057980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e53f3-12db-4b2e-8f75-94182aed47cd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.063412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.064258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.064535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.065107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] PATH_INFO: `/v3/projects/e599af8fd2004feb828606429d7995e0/users/55dd393786744f2eb98de926147a8774/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.086487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e599af8fd2004feb828606429d7995e0, user_id=55dd393786744f2eb98de926147a8774, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.089966 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2926/5781] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:03.089966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.089966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.089966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.090383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.126983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2856/5782] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:02 2022] PUT /identity/v3/projects/e599af8fd2004feb828606429d7995e0/users/55dd393786744f2eb98de926147a8774/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:03.129304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.136675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:03.139073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.211635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5b5ab-919c-47e5-93a2-e24c5242dd94 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.216433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.217027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.217302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.217573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.218188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.220400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.222069 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2927/5783] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:03.223684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.239492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afb2b-013a-40a4-91a8-514c26ed844c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.249957 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2857/5784] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:03.251447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.254761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.256302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.256969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.257358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.257615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.258411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.260647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.275695 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2928/5785] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:03.276770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.280890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d2b64-fd1b-446e-a448-3eb5394e3016 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.285394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.286297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.286755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.287082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] PATH_INFO: `/v3/users/e7c6d5b5e4d7454da96475a419d81b82` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.288406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c6d5b5e4d7454da96475a419d81b82)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.290470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.291346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b582600-d73f-4398-a02e-85821e2de94f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.305938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.306479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.306828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.307052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.307732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:03.307976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.309788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24454a02-c6a2-4d3b-a108-32a9e8340b04 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.336797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2929/5786] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:03.344704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.344991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.345270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.360805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c6d5b5e4d7454da96475a419d81b82'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.362096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c6d5b5e4d7454da96475a419d81b82'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.372347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c6d5b5e4d7454da96475a419d81b82'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.378205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] MFA Rules not processed for user `55dd393786744f2eb98de926147a8774`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:03.380473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.381798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c6d5b5e4d7454da96475a419d81b82'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.382651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c6d5b5e4d7454da96475a419d81b82'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.407587 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2858/5787] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:03 2022] DELETE /identity/v3/users/e7c6d5b5e4d7454da96475a419d81b82 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:03.408560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.410787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.440047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:03.454724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:03.462914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2930/5788] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:03.464409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.465880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.479917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baeab1-93c9-4e5e-bce4-26352faf7ce1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.480512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab451070-0a9d-4907-8919-ff2024d5844f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.484955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.485552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.485679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.485827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.486097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.486294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.486565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.486809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:03.486920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.487063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.487497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.489638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.489772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.503213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2931/5789] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:03.503213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.503639 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2859/5790] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:03.503804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.503904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.505813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.516734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f9540-2d94-45b0-a4bb-4909b66d8c33 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.518205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f50600-9a76-4419-9fb0-6b4b4bcba62d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.522489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.523721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.524005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.524277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] PATH_INFO: `/v3/projects/0ad8713cc3a0464699bc316e93f94d28/users/fef892bec8a7455b99a5f52f94a145bc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.530721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.531366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.531637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.531909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] PATH_INFO: `/v3/projects/d6f9abd4cfb1462da57590aa5f311707` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.533037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f9abd4cfb1462da57590aa5f311707)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.535829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.540600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad8713cc3a0464699bc316e93f94d28, user_id=fef892bec8a7455b99a5f52f94a145bc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.543279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.559195 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2932/5791] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:03 2022] PUT /identity/v3/projects/0ad8713cc3a0464699bc316e93f94d28/users/fef892bec8a7455b99a5f52f94a145bc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:03.560058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.561705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.590431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f9abd4cfb1462da57590aa5f311707', 'request_id': 'req-c51ee920-17e2-455c-894f-e3f18c762c6c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.596166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f9abd4cfb1462da57590aa5f311707', 'request_id': 'req-c51ee920-17e2-455c-894f-e3f18c762c6c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.606308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f9abd4cfb1462da57590aa5f311707', 'request_id': 'req-c51ee920-17e2-455c-894f-e3f18c762c6c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.618437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] The token cache is being invalidate because project d6f9abd4cfb1462da57590aa5f311707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:03.618757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.619410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c51ee920-17e2-455c-894f-e3f18c762c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:03.620904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2860/5792] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:03 2022] DELETE /identity/v3/projects/d6f9abd4cfb1462da57590aa5f311707 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:03.624363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.624363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.624363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.635206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b1ecb-675e-481c-ae57-17ec3d297ab2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.641711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:03.643069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.700773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.701384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.701665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.701936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.702685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2861/5793] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:03.702806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.703322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.704638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.707176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.720420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ab368-4ff4-4725-8c45-af1fd4ec2ea9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.726352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.727032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.727319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.727593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.728215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.730133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.736634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2933/5794] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:03.736634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.736634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.759368 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2862/5795] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:03.760699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.762372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.784515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:03.804990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:03.811198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cdbe7-b897-41fb-a084-194d9e1f9b2b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.831414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaefd96-90e5-40a3-bf29-64849f6c3920 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.836258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.836866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.837150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.837627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.838189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.840298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.875108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2863/5796] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/projects => generated 361 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:03.875960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.879847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.881162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.881837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.882153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.882649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.883484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.885726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18824f0f-d6c2-45ff-b318-b97081c912d0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.902666 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2934/5797] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:03.904135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.904420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.904690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.926764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:03.928102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.949196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8d724-9aad-4782-9f27-2f16ec6cd558 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:03.954165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:03.954816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:03.955094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:03.955460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:03.956059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:03.957718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:03.967884 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2864/5798] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:03.968520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.970180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:03.989147 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2935/5799] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:03.989862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:03.991167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.002791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49936401-50d2-4f35-bfe8-92f1d4d425f8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.007798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.008426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.008700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.009401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.010090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.013665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.036372 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2936/5800] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:03 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:04.037391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.037871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15023bc-ef89-47e5-a0cd-b341ee2aa5b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.041792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.042595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.042852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.043123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.047749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce913a-7530-4c49-b5f4-8d1889051f1f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.053295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.054059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.054683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.054958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.055074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.055234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.055838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.057707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.065178 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2865/5801] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:04.066222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.077150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f331d7-02d6-49f6-a186-8da83286ead9 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.082842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2937/5802] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:04.082842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.085727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.086382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.086692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.087012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.088472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.089540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.092294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94faf-b34e-444f-98e0-a34c6a65d154 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.098203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.099105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.100130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.100406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] PATH_INFO: `/v3/projects/1ce0317995ab4b69a98fcbbd84e901fb/users/743d13df699144278c4ca0e2b92e40b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.115479 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2866/5803] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/users => generated 281 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:04.116208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.123361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce0317995ab4b69a98fcbbd84e901fb, user_id=743d13df699144278c4ca0e2b92e40b2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.126207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e40c9d-7bcc-455f-b8f2-36d64cc285a1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.127395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5a664-1d3d-430b-8995-a21c0f5940eb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.133864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.135220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.135609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.136004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] PATH_INFO: `/v3/projects/0ad8713cc3a0464699bc316e93f94d28/users/fef892bec8a7455b99a5f52f94a145bc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.138095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad8713cc3a0464699bc316e93f94d28, user_id=fef892bec8a7455b99a5f52f94a145bc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.141224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2938/5804] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:04 2022] PUT /identity/v3/projects/1ce0317995ab4b69a98fcbbd84e901fb/users/743d13df699144278c4ca0e2b92e40b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.141809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.143022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.143498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.144076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.157418 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2867/5805] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:04 2022] PUT /identity/v3/projects/0ad8713cc3a0464699bc316e93f94d28/users/fef892bec8a7455b99a5f52f94a145bc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.158063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.169817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:04.170837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.178751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.210448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.240460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a897a5-ae37-4395-b10b-c18792f5f5e2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.243507 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2939/5806] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:04.244333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.248940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.249775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.250074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.250436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.251206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.252826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.256747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7ca85-41eb-4b9e-928c-daac4358f09e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.262872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.263737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.264121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.264593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.264830 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2868/5807] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.268424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.268671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.268782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413f4c4-fed4-49fc-9f58-064b55669b0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.286345 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2940/5808] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.287916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.288204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.288479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.290768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78d709-dd74-409a-98f8-28d656e0047b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.295703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.296310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.297324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.297324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.297552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.299245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d0f8-78a0-424b-a8d9-fa9567e1f92c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.310490 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2869/5809] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.311838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.312225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.312601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.318272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] MFA Rules not processed for user `743d13df699144278c4ca0e2b92e40b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:04.319541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.340765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] MFA Rules not processed for user `fef892bec8a7455b99a5f52f94a145bc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:04.341976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.388365 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2941/5810] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:04.389351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.400440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823f256-4fd7-4094-85bb-dfb60fa521af None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.405321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.405900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.406172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.406435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.407107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:04.407404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.408965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.414614 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2870/5811] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:04.415608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.427219 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2942/5812] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:04.427219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.427503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f6c97-c512-45a0-9a88-2fe0618db98a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.427605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.430863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.431462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.431742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.432012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.432645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.434244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.440885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1cefe-cccd-4cf6-a1d8-37c1cbbbdfb7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.444744 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2871/5813] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.445495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.445619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.446116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.446397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.446706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] PATH_INFO: `/v3/projects/5d9195ac8aaa41d7b558ae60158e561b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.447820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9195ac8aaa41d7b558ae60158e561b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.449951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.456006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb685-888c-40cf-b9e2-7c892cf06859 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.461537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.462389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.463130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.463434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] PATH_INFO: `/v3/projects/09a9f7bf1fc246eda820151e97653b2c/users/38ddb93a51504e32b2816ee6b6733956/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.480002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a9f7bf1fc246eda820151e97653b2c, user_id=38ddb93a51504e32b2816ee6b6733956, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.482924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.496154 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2872/5814] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:04 2022] PUT /identity/v3/projects/09a9f7bf1fc246eda820151e97653b2c/users/38ddb93a51504e32b2816ee6b6733956/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.499108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.499275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9195ac8aaa41d7b558ae60158e561b', 'request_id': 'req-2e31dc72-ba2f-4864-b6fb-76d26d218439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:04.505602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9195ac8aaa41d7b558ae60158e561b', 'request_id': 'req-2e31dc72-ba2f-4864-b6fb-76d26d218439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:04.512505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9195ac8aaa41d7b558ae60158e561b', 'request_id': 'req-2e31dc72-ba2f-4864-b6fb-76d26d218439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:04.526980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] The token cache is being invalidate because project 5d9195ac8aaa41d7b558ae60158e561b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:04.527409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:04.528095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:04.529611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2943/5815] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:04 2022] DELETE /identity/v3/projects/5d9195ac8aaa41d7b558ae60158e561b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.530789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.532103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.551157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.566877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.569621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c8c0-14da-48bc-84b8-7b22c807ee6a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.576651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.577405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.577794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.578080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.578761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.580364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.591103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2873/5816] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.591885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.593615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.599642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31dc72-ba2f-4864-b6fb-76d26d218439 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.668393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9caa557423fd43069bec6eccce15169b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b18f80a424de46a089b6ba60df77cf48', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.669234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.669794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.670594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] PATH_INFO: `/v3/users/9caa557423fd43069bec6eccce15169b/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.671364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] RBAC: Authorizing `identity:list_user_projects(user_id=9caa557423fd43069bec6eccce15169b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.673287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.676992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56c55d-9d12-424a-9464-aecb9a1acc27 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.682387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.683089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.683373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.683642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.684260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.686105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.716756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2944/5817] 10.222.0.52 () {58 vars in 1248 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/users/9caa557423fd43069bec6eccce15169b/projects => generated 515 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:04.717570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.719728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2874/5818] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/projects => generated 358 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:04.720969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.723872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.725969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.736504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f68fbf-d34e-489b-8cce-a4cda6414bab tempest-IdentityV3ProjectsTest-102499592 tempest-IdentityV3ProjectsTest-102499592-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.795609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a2f6d-68f3-4ea7-884d-452794b6ca2c tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.798245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.798898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.799221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.799490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.800109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.800892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.801600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.801973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.802254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] PATH_INFO: `/v3/projects/50f7cb0bd6bd410e994ec45db40340ca/users/b3ca8ed4b98645ad9959fef8bacbdcd6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.802810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.818398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f7cb0bd6bd410e994ec45db40340ca, user_id=b3ca8ed4b98645ad9959fef8bacbdcd6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.821377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfff483-75dc-44b7-b06d-9730c6bc557e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.829344 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2945/5819] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/projects => generated 361 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:04.830293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.833552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.843207 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2875/5820] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:04 2022] PUT /identity/v3/projects/50f7cb0bd6bd410e994ec45db40340ca/users/b3ca8ed4b98645ad9959fef8bacbdcd6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.843207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.843207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.843207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.856924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:04.858204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.879368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.908331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:04.911986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048e9d1-018a-49dc-87b9-b6745160cc0b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.918079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.918841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.919121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.919494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.920230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.921835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29efdc7-88e4-4912-8a18-7fe12cb89c0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.931657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2876/5821] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:04.931657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:04.933095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.937105 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2946/5822] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:04.937105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.937105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.937105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.946894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b8c63-458e-45aa-8359-54d15cf44da0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:04.952009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:04.955703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:04.955985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:04.956254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] PATH_INFO: `/v3/projects/68da2093d56948859dc0eff5292ce484/users/2fc6521c92d94c37ac7bf4049eca7143/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:04.968399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] MFA Rules not processed for user `9caa557423fd43069bec6eccce15169b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:04.969615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:04.978202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68da2093d56948859dc0eff5292ce484, user_id=2fc6521c92d94c37ac7bf4049eca7143, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:04.981176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:04.996185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2877/5823] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:04 2022] PUT /identity/v3/projects/68da2093d56948859dc0eff5292ce484/users/2fc6521c92d94c37ac7bf4049eca7143/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:04.997089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.037623 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2947/5824] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.038623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.049045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53016213-b6b6-4a81-9fe0-b24caea0e1f8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.065874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07e47-193d-45cb-b157-8bf43332be3b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.070535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.071133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.071411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.071685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.072304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.074144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580da98-df7e-4ff3-9eb7-da00bd7fee52 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.101444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2878/5825] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:04 2022] POST /identity/v3/users => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:05.103114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b806c-57e1-4496-93f9-5761067da6e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.103397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b806c-57e1-4496-93f9-5761067da6e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.103796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b806c-57e1-4496-93f9-5761067da6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.107613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.108230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.108510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.108781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.109400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.111406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9239d-a80f-4e22-bcb1-dfba6b266f17 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.133017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f89b806c-57e1-4496-93f9-5761067da6e5 None None] MFA Rules not processed for user `b3ca8ed4b98645ad9959fef8bacbdcd6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.134322 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2948/5826] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:05.134639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f89b806c-57e1-4496-93f9-5761067da6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.135462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.135799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.136016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.156248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.157609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.166579 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2949/5827] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.169931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.177426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a663-f70e-4da8-a308-5bb8d35ad258 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.182729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.183463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.183743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.184011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] PATH_INFO: `/v3/projects/09a9f7bf1fc246eda820151e97653b2c/users/38ddb93a51504e32b2816ee6b6733956/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.186574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a9f7bf1fc246eda820151e97653b2c, user_id=38ddb93a51504e32b2816ee6b6733956, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.189212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777aa27-f337-4879-b8b4-2b6c8724bad6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.199850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2950/5828] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:05 2022] PUT /identity/v3/projects/09a9f7bf1fc246eda820151e97653b2c/users/38ddb93a51504e32b2816ee6b6733956/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:05.201092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.201372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.201638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.211278 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2879/5829] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.211896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.212183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.212453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.233918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] MFA Rules not processed for user `2fc6521c92d94c37ac7bf4049eca7143`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.237397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.243522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] MFA Rules not processed for user `9caa557423fd43069bec6eccce15169b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.250352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.276194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.297155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.models.token_model [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] User 9caa557423fd43069bec6eccce15169b has no access to project 1ce0317995ab4b69a98fcbbd84e901fb {{(pid=99133) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} Sep 24 21:12:05.301649 ncdv-master-1 devstack@keystone.service[99133]: WARNING keystone.server.flask.application [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:12:05.302986 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2880/5830] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 92 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:12:05.307563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.310398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.318171 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2951/5831] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.319372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.385381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f171d-9099-4af4-bb7a-87fca631b008 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.386760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd0bf-faa8-41e4-bb35-9cf259913c1f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.390137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.390958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.391238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.391359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.391496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.391962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.392121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.392230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.392501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.393213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:05.393499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.393752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.395665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0fde1-0475-4f95-b41c-6cad02d0b1e2 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.405211 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2952/5832] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:05.405211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.416611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bab2-e57a-45f9-928f-b25063753251 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.423083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.423810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.424160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.424500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.425293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:05.426392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.428215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.433122 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2881/5833] 10.222.0.52 () {58 vars in 1157 bytes} [Sat Sep 24 21:12:05 2022] GET /identity/v3/users?domain_id=default => generated 8294 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:05.433122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.433122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.433122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.442812 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2953/5834] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:05.444156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.454489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970d85-41f7-4771-b00a-5d246c9f96ab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.459875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.460464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.460743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.461014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.461678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.462101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] MFA Rules not processed for user `38ddb93a51504e32b2816ee6b6733956`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.463348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.463618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.490936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2954/5835] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:05.491758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.493449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.537523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2882/5836] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.538445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.548545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d7c50-c80f-4c98-8078-f44a71a10d9d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.565522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5349a47-2e50-450a-95d0-17cd7bfd568e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.570939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.571347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.571628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.571898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] PATH_INFO: `/v3/users/9caa557423fd43069bec6eccce15169b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.573031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9caa557423fd43069bec6eccce15169b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.575032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.608460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.609207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.609496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.609773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] PATH_INFO: `/v3/users/2fc6521c92d94c37ac7bf4049eca7143` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.610959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc6521c92d94c37ac7bf4049eca7143)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.611379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.612981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.626680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9caa557423fd43069bec6eccce15169b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.627475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9caa557423fd43069bec6eccce15169b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.636060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9caa557423fd43069bec6eccce15169b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.650989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9caa557423fd43069bec6eccce15169b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.651761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9caa557423fd43069bec6eccce15169b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.651927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.666275 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2955/5837] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:05 2022] DELETE /identity/v3/users/9caa557423fd43069bec6eccce15169b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:05.667270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.668733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.672474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc6521c92d94c37ac7bf4049eca7143'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.673145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc6521c92d94c37ac7bf4049eca7143'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.680122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc6521c92d94c37ac7bf4049eca7143'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.687537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc6521c92d94c37ac7bf4049eca7143'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.688266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc6521c92d94c37ac7bf4049eca7143'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.707630 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2883/5838] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:05 2022] DELETE /identity/v3/users/2fc6521c92d94c37ac7bf4049eca7143 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:05.708430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.709786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.736682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c35ad2-6a3c-4019-818b-875731af11c7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.776843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1f11b-953f-4b81-b86a-759070c5c2d5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.781756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.782385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.782715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.782993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/users/82edbc54490e4ac6883052260767ec8a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.797143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:update_user(user_id=82edbc54490e4ac6883052260767ec8a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.799121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.799425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.800039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.800833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.801105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.801743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.803507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.817259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2956/5839] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:05.818166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.820073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.831041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.833284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442310ef-8dec-4195-8342-8c80b891f176 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.838701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.839360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.839685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.840054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.840499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invalidating the token cache because user 82edbc54490e4ac6883052260767ec8a was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:05.840635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.840789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.841513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:05.842601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.843368 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2884/5840] 10.222.0.52 () {60 vars in 1244 bytes} [Sat Sep 24 21:12:05 2022] PATCH /identity/v3/users/82edbc54490e4ac6883052260767ec8a => generated 485 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:05.844115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.845407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.858777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ed184-440e-4a34-b317-ef0b7033bceb tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.864604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.865478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.865756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.866268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.866681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:05.868609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c8353-c54b-4bd1-88e1-d7a6425bdc61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:05.875906 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2957/5841] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:05.875906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:05.875906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.891318 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2885/5842] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:05.895596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662880-3fe8-418c-abbc-cf542723350e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.895887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662880-3fe8-418c-abbc-cf542723350e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.896164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e662880-3fe8-418c-abbc-cf542723350e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.912193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.915891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7e662880-3fe8-418c-abbc-cf542723350e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:05.917287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e662880-3fe8-418c-abbc-cf542723350e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.941987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33406011-c399-42ee-91b1-aa95e12f7b29 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:05.945835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:05.946442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.946756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.947030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:05.948809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:05.968282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=5,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:05.985834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2886/5843] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:05.987052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:05.987511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:05.988268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.010243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:06.011654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.018476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.020503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91193109-a81d-42e8-a0fd-77f4cd1498a1 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.021082 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2887/5844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:06.023314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.030294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.031666 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2958/5845] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:06.033038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730e68-d880-429f-aac5-a3914b6543df None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.033336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730e68-d880-429f-aac5-a3914b6543df None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.033611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19730e68-d880-429f-aac5-a3914b6543df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.044161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0b25f-c528-4697-859d-2624882f0fea None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.052812 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.auth.plugins.core [None req-19730e68-d880-429f-aac5-a3914b6543df None None] User is disabled: 82edbc54490e4ac6883052260767ec8a: AssertionError: User is disabled: 82edbc54490e4ac6883052260767ec8a Sep 24 21:12:06.057527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.058583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.058883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.059288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] PATH_INFO: `/v3/projects/7d55af194b014318a32f36f36966b2c9/users/c358e4c85a7c4de3a9c7b465548b4681/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.060370 ncdv-master-1 devstack@keystone.service[99132]: WARNING keystone.server.flask.application [None req-19730e68-d880-429f-aac5-a3914b6543df None None] Authorization failed. The request you have made requires authentication. from 10.222.0.52: keystone.exception.Unauthorized: The request you have made requires authentication. Sep 24 21:12:06.062221 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2959/5846] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:06 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 30 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) Sep 24 21:12:06.063003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19730e68-d880-429f-aac5-a3914b6543df None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.067576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19730e68-d880-429f-aac5-a3914b6543df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.077371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d55af194b014318a32f36f36966b2c9, user_id=c358e4c85a7c4de3a9c7b465548b4681, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.080895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.085343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19730e68-d880-429f-aac5-a3914b6543df None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.090462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.091186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.091534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.091877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.092616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.094772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.112985 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2888/5847] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:06 2022] PUT /identity/v3/projects/7d55af194b014318a32f36f36966b2c9/users/c358e4c85a7c4de3a9c7b465548b4681/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:06.112985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.126145 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2960/5848] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:06 2022] POST /identity/v3/users => generated 292 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:06.127460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.130254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.187415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04627947-fcdd-4e50-8bfb-aa192ce182b7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.192858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.193463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.193774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.194052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.194724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.196336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.196883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ec4d8-ccab-4eb8-941f-86da333dfc0c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.212239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2889/5849] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:06.212239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.254362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.255072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.255352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.255626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.265536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.267026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.270435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9534271-c837-455c-9aae-ffa1736f13c8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.274719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.275457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.275741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.276027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.278299 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2961/5850] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:06.279167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.280765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.286980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.288441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575d468-d45e-4d1d-83e3-e8307631200a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.299563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2890/5851] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:06.301010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.301274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.301499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.302419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:06.335238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] MFA Rules not processed for user `c358e4c85a7c4de3a9c7b465548b4681`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:06.336683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.356858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab7746-83f8-4d8e-a353-c9b4926dcb27 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.361713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.362456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.362728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.363116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/users/82edbc54490e4ac6883052260767ec8a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.382488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:delete_user(user_id=82edbc54490e4ac6883052260767ec8a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.384772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.394197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:06.418717 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2891/5852] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:06.420043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.434006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdfc15-6ac5-4861-ab41-ba8f402d1c9b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.440938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.441776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.442468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.442468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.443601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.444750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.450024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:06.450757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:06.456206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2892/5853] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:06.457208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.458860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.464703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:06.479213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:06.480034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82edbc54490e4ac6883052260767ec8a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:06.497091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2962/5854] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:06 2022] DELETE /identity/v3/users/82edbc54490e4ac6883052260767ec8a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:06.497709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.499434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.540922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e637c55-0b3a-4a0d-a1ab-2cabea56f04d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.604873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651d90-5e8a-424c-8024-e97a5d2e2f5f tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.614104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.615381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.615872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.616316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.616498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] PATH_INFO: `/v3/projects/74780edf6b114f7ca7a9020c041482c3/users/881b1fad2940439988bd4d9c7fa2301b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.616935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.617213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.617483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.619734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.643612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74780edf6b114f7ca7a9020c041482c3, user_id=881b1fad2940439988bd4d9c7fa2301b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.650280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f98da-f405-411a-b48a-54998eeea0c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.673015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2963/5855] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:06 2022] PUT /identity/v3/projects/74780edf6b114f7ca7a9020c041482c3/users/881b1fad2940439988bd4d9c7fa2301b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:06.674490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.675910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.675910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.692138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:06.700890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] MFA Rules not processed for user `c358e4c85a7c4de3a9c7b465548b4681`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:06.702749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.712254 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2964/5856] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:06 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:12:06.713926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.715564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.728773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.732972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.744989 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2893/5857] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:06.746076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.747868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.817348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:06.821875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7c9c-8f28-4d32-84ed-dfcff661787b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.827830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.828460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.828744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.829135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.829719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.832615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.843334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115676-18d4-4566-9ab1-68f999f78e9c service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.847020 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2965/5858] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:06.849001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.861153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba313a6-cac4-4729-9e02-356bff74f5ef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.866647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.867250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.867535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.867818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.868452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.870152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3320b35-335e-41a1-8599-450a9d94f366 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.881851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2966/5859] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:06.883104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.883430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.883670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.904185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] MFA Rules not processed for user `c358e4c85a7c4de3a9c7b465548b4681`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:06.905703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.910012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2967/5860] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:06 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:12:06.910739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:06.912284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:06.919514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.920157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.920507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.920712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/projects/ec0db243dddb4fa6b60e97400b751d26` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.921971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ec0db243dddb4fa6b60e97400b751d26)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.923893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.928376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c600a8c-bb5b-4526-b5b6-01eb44f7739a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:06.933203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:06.933840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:06.934474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:06.934474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] PATH_INFO: `/v3/projects/68da2093d56948859dc0eff5292ce484` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:06.935661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68da2093d56948859dc0eff5292ce484)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:06.937577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:06.992909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0db243dddb4fa6b60e97400b751d26', 'request_id': 'req-8b425aec-c236-4ff5-b587-cafb12657769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.000519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0db243dddb4fa6b60e97400b751d26', 'request_id': 'req-8b425aec-c236-4ff5-b587-cafb12657769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.002837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68da2093d56948859dc0eff5292ce484', 'request_id': 'req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.009425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0db243dddb4fa6b60e97400b751d26', 'request_id': 'req-8b425aec-c236-4ff5-b587-cafb12657769'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.009856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68da2093d56948859dc0eff5292ce484', 'request_id': 'req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.016472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68da2093d56948859dc0eff5292ce484', 'request_id': 'req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.021362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] The token cache is being invalidate because project ec0db243dddb4fa6b60e97400b751d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:07.021627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.022282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.023839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2894/5861] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:06 2022] DELETE /identity/v3/projects/ec0db243dddb4fa6b60e97400b751d26 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.024684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.029404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.029588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] The token cache is being invalidate because project 68da2093d56948859dc0eff5292ce484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:07.030137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.030821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.032530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2968/5862] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:06 2022] DELETE /identity/v3/projects/68da2093d56948859dc0eff5292ce484 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.033145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.035060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.102137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19d59-93b3-46ed-9de8-ed87f84cbf8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.105034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b425aec-c236-4ff5-b587-cafb12657769 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.107160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.108300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.108587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.108865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] PATH_INFO: `/v3/projects/ab991a36186f40b2bee70625b3f7424c/users/325f5dc462514e8e85d5adf9d433cb40/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.112273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.115088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.115807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.116449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.116807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] PATH_INFO: `/v3/projects/b18f80a424de46a089b6ba60df77cf48` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.118132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18f80a424de46a089b6ba60df77cf48)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.121212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.127866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab991a36186f40b2bee70625b3f7424c, user_id=325f5dc462514e8e85d5adf9d433cb40, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.132027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.146908 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2969/5863] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:07 2022] PUT /identity/v3/projects/ab991a36186f40b2bee70625b3f7424c/users/325f5dc462514e8e85d5adf9d433cb40/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.147692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.149328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.165146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.181999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18f80a424de46a089b6ba60df77cf48', 'request_id': 'req-0d666b5c-96e5-49b8-8718-4ef3af5348a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.187548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18f80a424de46a089b6ba60df77cf48', 'request_id': 'req-0d666b5c-96e5-49b8-8718-4ef3af5348a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.195449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18f80a424de46a089b6ba60df77cf48', 'request_id': 'req-0d666b5c-96e5-49b8-8718-4ef3af5348a4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.207748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] The token cache is being invalidate because project b18f80a424de46a089b6ba60df77cf48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:07.208042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.208694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.210198 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2895/5864] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:07 2022] DELETE /identity/v3/projects/b18f80a424de46a089b6ba60df77cf48 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.211079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.212482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.218553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72f74b-d0af-4e2e-ac77-a3155eb8ebd6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.274886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d666b5c-96e5-49b8-8718-4ef3af5348a4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.276443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.277104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.277382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.277767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] PATH_INFO: `/v3/projects/74780edf6b114f7ca7a9020c041482c3/users/881b1fad2940439988bd4d9c7fa2301b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.279179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.279697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74780edf6b114f7ca7a9020c041482c3, user_id=881b1fad2940439988bd4d9c7fa2301b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.280064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.280299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.280571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.282484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.282762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.291121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorizing `identity:check_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.292801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1d0c7-52df-4deb-8a33-8580dd028b12 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.294438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2970/5865] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:07 2022] PUT /identity/v3/projects/74780edf6b114f7ca7a9020c041482c3/users/881b1fad2940439988bd4d9c7fa2301b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.299462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.299706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2896/5866] 10.222.0.52 () {56 vars in 1264 bytes} [Sat Sep 24 21:12:07 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) Sep 24 21:12:07.301435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.301776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f8a4-cdb6-4b88-a75a-c7aa9a9505ac None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.302380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f8a4-cdb6-4b88-a75a-c7aa9a9505ac None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.303804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f8a4-cdb6-4b88-a75a-c7aa9a9505ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.325733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dc74f8a4-cdb6-4b88-a75a-c7aa9a9505ac None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:07.326985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dc74f8a4-cdb6-4b88-a75a-c7aa9a9505ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.370271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce79d4a-759f-481a-ae0d-61687eb4c004 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.374871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fef892bec8a7455b99a5f52f94a145bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad8713cc3a0464699bc316e93f94d28', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.375494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.375742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.376012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.376635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.378614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.389270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2897/5867] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:07.390697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.390984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.391258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Could not find domain: 9eee304d503142419df3585822864fd8.: keystone.exception.DomainNotFound: Could not find domain: 9eee304d503142419df3585822864fd8. Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9eee304d503142419df3585822864fd8. Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:12:07.391980 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user Sep 24 21:12:07.393152 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:12:07.399047 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9eee304d503142419df3585822864fd8. Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2971/5868] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:07 2022] POST /identity/v3/users => generated 112 bytes in 99 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.400322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.413387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa009a-059a-4b23-adca-5c0749f9f762 tempest-UsersNegativeTest-662933026 tempest-UsersNegativeTest-662933026-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.418188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.418821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.419112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.419361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] PATH_INFO: `/v3/users/743d13df699144278c4ca0e2b92e40b2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.420472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743d13df699144278c4ca0e2b92e40b2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.422483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.424322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] MFA Rules not processed for user `325f5dc462514e8e85d5adf9d433cb40`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:07.425610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.444435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.471688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743d13df699144278c4ca0e2b92e40b2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.472984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743d13df699144278c4ca0e2b92e40b2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.474212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.481274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743d13df699144278c4ca0e2b92e40b2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.494306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743d13df699144278c4ca0e2b92e40b2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.494998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1e13c09d-e9d1-4e2d-bfca-87a2ecfed9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743d13df699144278c4ca0e2b92e40b2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.496717 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2898/5869] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:07.500899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.502235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.508106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2972/5870] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:07 2022] DELETE /identity/v3/users/743d13df699144278c4ca0e2b92e40b2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.509731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c205-ba94-4b52-a3ed-ef15e8a666df None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.510017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c205-ba94-4b52-a3ed-ef15e8a666df None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.510285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c205-ba94-4b52-a3ed-ef15e8a666df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.541478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3566c205-ba94-4b52-a3ed-ef15e8a666df None None] MFA Rules not processed for user `881b1fad2940439988bd4d9c7fa2301b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:07.542966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3566c205-ba94-4b52-a3ed-ef15e8a666df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.571661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79be5a-1196-4dc9-97a8-69eb6f5b6b38 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.636417 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2973/5871] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:07.637700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.638301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.638807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.643222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.645140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.645963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.646424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.648535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.658282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.660994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.668815 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2899/5872] 10.222.0.52 () {56 vars in 1263 bytes} [Sat Sep 24 21:12:07 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 21:12:07.669975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.671256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:07.672471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.673107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.760316 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2974/5873] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:07.761269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.763091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.768876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82673e94-08c5-4bd4-86f8-213212fe9346 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.778459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.779129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.779412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.780396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] PATH_INFO: `/v3/users/5d572a1f774345018fd0fbaf161a6fdf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.781438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d572a1f774345018fd0fbaf161a6fdf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.783552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.819227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.845369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:07.854052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d572a1f774345018fd0fbaf161a6fdf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.854843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d572a1f774345018fd0fbaf161a6fdf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.864138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0659d2-d087-40ab-9c56-38e4233693a6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.866011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d572a1f774345018fd0fbaf161a6fdf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.868494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.869141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.869424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.869694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.871524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.874355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d572a1f774345018fd0fbaf161a6fdf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.875063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d572a1f774345018fd0fbaf161a6fdf'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:07.895702 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2900/5874] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:07 2022] DELETE /identity/v3/users/5d572a1f774345018fd0fbaf161a6fdf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:07.895890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.897166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.938329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:07.940405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:07.949987 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2975/5875] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:07.951049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:07.952446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:07.959460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdd68d-2d46-422d-a68b-0ccea7c81598 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.963194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:07.963873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:07.964127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:07.964310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:07.965041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccf55f-9a60-4b28-bd11-14be00a9053e service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:07.966469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.030379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.031170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.031563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.031846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.034021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.045565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.047190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.055264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.059559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.069222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2976/5876] 10.222.0.52 () {56 vars in 1266 bytes} [Sat Sep 24 21:12:07 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:08.070147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.074820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2901/5877] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:08.075965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.077741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.083368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.172737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01113322-7f80-49c8-afa7-6bacfb193496 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.180112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.180815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.181098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.181586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] PATH_INFO: `/v3/projects/1ce0317995ab4b69a98fcbbd84e901fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.182856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce0317995ab4b69a98fcbbd84e901fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.185408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.197282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:08.245201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5355cf-e1ed-4901-ba72-db4dce13c63d tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.252930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.254807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.255691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.256658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.257621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:08.258142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.261588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77054b83-41fb-434a-8d23-5fddbe3b58a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.285674 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2977/5878] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:08.285674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.285674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.285674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.287937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce0317995ab4b69a98fcbbd84e901fb', 'request_id': 'req-3adf6fa3-fa08-4037-8759-e6b5d12086ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.297669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:08.305633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce0317995ab4b69a98fcbbd84e901fb', 'request_id': 'req-3adf6fa3-fa08-4037-8759-e6b5d12086ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.318426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:08.320259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.327083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce0317995ab4b69a98fcbbd84e901fb', 'request_id': 'req-3adf6fa3-fa08-4037-8759-e6b5d12086ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.353490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] The token cache is being invalidate because project 1ce0317995ab4b69a98fcbbd84e901fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:08.353894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.355800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.363802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2902/5879] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:08 2022] DELETE /identity/v3/projects/1ce0317995ab4b69a98fcbbd84e901fb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:08.364784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.366441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.443294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2978/5880] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:08.448233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.448972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.465153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683c52-6bec-47b6-a220-c3b38166cadd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.470160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.470824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.471233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.471604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.472192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.474266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35a084-eeac-4700-8c27-93fec065059b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.507044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6fa3-fa08-4037-8759-e6b5d12086ab admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.522028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.522707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.522983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.523253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.528750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.529985 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2979/5881] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:08.531315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.531602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.531879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') Sep 24 21:12:08.547576 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 24 21:12:08.548640 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:12:08.550435 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2903/5882] 10.222.0.52 () {56 vars in 1264 bytes} [Sat Sep 24 21:12:08 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) Sep 24 21:12:08.551255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.552330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.560472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:08.561947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.663321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72335425-020a-45fd-9af9-451d82f35918 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.666106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2980/5883] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:08.667486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.683006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75645a12-58d5-41bb-be3d-3136a0f9db5d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.685761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.686539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.687081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.687458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.688087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.690079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eff8c3-c963-4ef0-81f4-ce24b54f324a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.725060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2981/5884] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/users => generated 289 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:08.726727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.727051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.727558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.749209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] MFA Rules not processed for user `c358e4c85a7c4de3a9c7b465548b4681`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:08.750731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.757814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2982/5885] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 21:12:08.757814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.758613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.776732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03e1c5-ace1-4a7a-8dae-e162b8491bac None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.783573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.785226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.786090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.786909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] PATH_INFO: `/v3/users/55dd393786744f2eb98de926147a8774` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.788863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dd393786744f2eb98de926147a8774)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.791260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.799960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.801571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.801960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.802341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.815481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.817703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc629a41-9b8c-4362-b24b-a8ea6a9c702f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.824854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:08.833089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2904/5886] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:08.834451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.835177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.835774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.856671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dd393786744f2eb98de926147a8774'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.857405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dd393786744f2eb98de926147a8774'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.859160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:08.860392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.864086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:08.870556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dd393786744f2eb98de926147a8774'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.870743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2905/5887] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:08.871895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.875155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.889082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5111cd-3dde-41ba-ad64-110b4d713180 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.890243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dd393786744f2eb98de926147a8774'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.891052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dd393786744f2eb98de926147a8774'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:08.896076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.896624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.896991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.897339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.898101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:08.900023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:08.905364 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2983/5888] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:08 2022] DELETE /identity/v3/users/55dd393786744f2eb98de926147a8774 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:08.906208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.907698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.914793 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2906/5889] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:08.914793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:08.916969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.930907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff582a-5d7f-40cf-9a75-358bfda7f968 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.981616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45fd0f-7ed8-43e5-afa8-bf446e385ea3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:08.987651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.989142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.989520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.989892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:08.992775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:08.997634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:08.998320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:08.998715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:08.998992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] PATH_INFO: `/v3/users/b3ca8ed4b98645ad9959fef8bacbdcd6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.000361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ca8ed4b98645ad9959fef8bacbdcd6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.001264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.002597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.003589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.013861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2984/5890] 10.222.0.52 () {56 vars in 1263 bytes} [Sat Sep 24 21:12:08 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 21:12:09.013861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.026759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e9149-efbe-4e6e-b3c2-ffe825d3476e tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.032169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.032782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.033060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.033332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.033985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:09.034277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.037260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ebcff-7a6d-4047-84bc-79abebb5da35 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.049933 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2985/5891] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:09.051861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.052211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.052489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.071034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ca8ed4b98645ad9959fef8bacbdcd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.072164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ca8ed4b98645ad9959fef8bacbdcd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.073505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:09.074778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.084569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ca8ed4b98645ad9959fef8bacbdcd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.084836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2986/5892] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:09.085442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.087074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.099028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ca8ed4b98645ad9959fef8bacbdcd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.100152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ca8ed4b98645ad9959fef8bacbdcd6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.101882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631fbab-b8df-4f42-b06a-f7273a11d0b2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.107600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.108277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.108556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.108835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.109532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.111199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.123307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2907/5893] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:08 2022] DELETE /identity/v3/users/b3ca8ed4b98645ad9959fef8bacbdcd6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:09.123307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.123638 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2987/5894] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:09.123750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.123850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.125337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.140348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f011e-a5ed-46e4-9da5-356366c6ce7c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.193057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60b2b9-d9df-477f-8e45-420ee0bbbc34 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.197556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.198148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.198425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.198719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.200616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.206219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.207533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.207864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.208210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.208913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.209568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.211020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38206-a0ce-433e-b9f6-cf24b6c7a473 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.211148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.213508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:09.234720 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2908/5895] 10.222.0.52 () {56 vars in 1266 bytes} [Sat Sep 24 21:12:09 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:09.234720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.237073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.256970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2988/5896] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:09.259719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.259850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.260342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.285582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:09.287045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.310780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd7fa0-2acb-485a-8d88-36449ced7118 tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.324561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:09.368083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.369171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.369540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.370028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.385813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2989/5897] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:09.386807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.399878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbce58-d8c5-426a-806d-48803e71aaa5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.406015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.406930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.407277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.407977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] PATH_INFO: `/v3/projects/8590b56a0c924f76ba18920e735e6b4d/users/57c163c13444454c9a23703c08a9cc5a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.424395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8590b56a0c924f76ba18920e735e6b4d, user_id=57c163c13444454c9a23703c08a9cc5a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.428108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.445264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2990/5898] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:09 2022] PUT /identity/v3/projects/8590b56a0c924f76ba18920e735e6b4d/users/57c163c13444454c9a23703c08a9cc5a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:09.446086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.452677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.454825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.466144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2909/5899] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:09.467083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.480683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dd76-1136-408e-b67a-302fc88ec48a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.533672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45e545-3183-49c9-989b-7e618b7e7b52 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.539220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c358e4c85a7c4de3a9c7b465548b4681', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7d55af194b014318a32f36f36966b2c9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.539860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.540120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.540419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.542288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.562151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 24 21:12:09.563294 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.check_revocation(token) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 Sep 24 21:12:09.564450 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:12:09.565575 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2991/5900] 10.222.0.52 () {56 vars in 1263 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 119 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:12:09.566279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.579325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.580090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.580439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.581801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.583316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.587921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56bd7-5b3a-47a2-abdc-616e498f9bca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.624967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2910/5901] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:09.636946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.640256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.640810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.669212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39ccb-1125-4feb-af79-ffc3c964a95a tempest-TokensV3Test-1287918637 tempest-TokensV3Test-1287918637-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.670352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:09.671711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.675307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.675957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.676295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.676569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.681287 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2911/5902] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:09.685480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.688342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.689893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.691762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.704328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa8a22-bed3-42e0-b7e3-2a37a61ea710 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.707725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2992/5903] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:09.708762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.711406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.712166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.712512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.712854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] PATH_INFO: `/v3/users/c358e4c85a7c4de3a9c7b465548b4681` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.714139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c358e4c85a7c4de3a9c7b465548b4681)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.716317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.720156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60774a9e-b9c2-403e-a871-b259399c8ef1 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.727230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.727847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.728150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.728531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.729176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.731726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.750679 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2993/5904] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:09.751753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.755117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.777146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06a1f1-fb7b-4e43-937a-fca20125953d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.782574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.783419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.784165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.784165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.797930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.802295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.811100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c358e4c85a7c4de3a9c7b465548b4681'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.812092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c358e4c85a7c4de3a9c7b465548b4681'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.817034 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2994/5905] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:09.817940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.822563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c358e4c85a7c4de3a9c7b465548b4681'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.832531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c358e4c85a7c4de3a9c7b465548b4681'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.834342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b76328f-f619-4e42-b8c9-84e9fddc4ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c358e4c85a7c4de3a9c7b465548b4681'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:09.837171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f8264-76ec-42ee-8cfe-28428f8c9b74 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.844605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:09.846518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.847074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.848347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.849500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:09.851537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:09.857535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2912/5906] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:09 2022] DELETE /identity/v3/users/c358e4c85a7c4de3a9c7b465548b4681 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:09.862168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c414-90c0-47be-94d7-24cf3f05b2e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:09.862750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c414-90c0-47be-94d7-24cf3f05b2e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:09.863251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c414-90c0-47be-94d7-24cf3f05b2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:09.867199 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2995/5907] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:09.872057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:09.874286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:09.897769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f770b-c381-4fc0-813e-36495acc0f6f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:09.904593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:09.905415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ea9c414-90c0-47be-94d7-24cf3f05b2e5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:09.906719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ea9c414-90c0-47be-94d7-24cf3f05b2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.016022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.017400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.017911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.018394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] PATH_INFO: `/v3/projects/c6f49a85802b4e3fa4fe8cd939afb037/users/73bfde6cb59b4f99be577627805610f6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.050255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f49a85802b4e3fa4fe8cd939afb037, user_id=73bfde6cb59b4f99be577627805610f6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.053153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.058876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2913/5908] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:10.090111 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2996/5909] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:09 2022] PUT /identity/v3/projects/c6f49a85802b4e3fa4fe8cd939afb037/users/73bfde6cb59b4f99be577627805610f6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:10.092412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.093025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.093395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.094754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.095179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.172848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] MFA Rules not processed for user `73bfde6cb59b4f99be577627805610f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:10.174099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.231979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd82e46-b38d-4e53-b594-dbdd60c8d52c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:10.238304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.239487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.240415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.241192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.282361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:10.342081 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2914/5910] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:10.344369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.353068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.359091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.361474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.377806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2997/5911] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:10.381965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.385626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.396977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de052c0-77ec-4f01-864b-1160d9780c4c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:10.411772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.412429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.415262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.415583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] PATH_INFO: `/v3/projects/1d08d749ed5c432ca9776526cd9aebd7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.417946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d08d749ed5c432ca9776526cd9aebd7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.422962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.486665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:10.539051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d08d749ed5c432ca9776526cd9aebd7', 'request_id': 'req-4bfdd3d8-1210-41c1-91f8-b51107367818'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.547938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6678c-ebed-4526-a560-0f3d9caef754 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:10.553955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d08d749ed5c432ca9776526cd9aebd7', 'request_id': 'req-4bfdd3d8-1210-41c1-91f8-b51107367818'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.572793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d08d749ed5c432ca9776526cd9aebd7', 'request_id': 'req-4bfdd3d8-1210-41c1-91f8-b51107367818'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.592951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] The token cache is being invalidate because project 1d08d749ed5c432ca9776526cd9aebd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:10.593414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.595178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.599211 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2915/5912] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:10 2022] DELETE /identity/v3/projects/1d08d749ed5c432ca9776526cd9aebd7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:10.600815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.603789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.632162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.633768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.634611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.635199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.637736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.758337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdd3d8-1210-41c1-91f8-b51107367818 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:10.774073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.774845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.775255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.775532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] PATH_INFO: `/v3/projects/e599af8fd2004feb828606429d7995e0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.775712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.778430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.779187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e599af8fd2004feb828606429d7995e0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.781587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.795421 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2998/5913] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:10.796431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.798931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.820912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671694b-1baa-4a2f-9ac6-279fc042fbce service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:10.832841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:10.834685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:10.834979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:10.835261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] PATH_INFO: `/v3/users/fef892bec8a7455b99a5f52f94a145bc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:10.839496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef892bec8a7455b99a5f52f94a145bc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:10.844076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:10.899171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e599af8fd2004feb828606429d7995e0', 'request_id': 'req-4d8a4596-826f-4b06-ad08-e121679001b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.914878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e599af8fd2004feb828606429d7995e0', 'request_id': 'req-4d8a4596-826f-4b06-ad08-e121679001b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.941317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e599af8fd2004feb828606429d7995e0', 'request_id': 'req-4d8a4596-826f-4b06-ad08-e121679001b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.944237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:10.966175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] The token cache is being invalidate because project e599af8fd2004feb828606429d7995e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:10.967882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.968856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.976723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2916/5914] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:10 2022] DELETE /identity/v3/projects/e599af8fd2004feb828606429d7995e0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:10.977766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:10.979292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:10.981460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef892bec8a7455b99a5f52f94a145bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:10.982304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef892bec8a7455b99a5f52f94a145bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.002526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef892bec8a7455b99a5f52f94a145bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.028264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef892bec8a7455b99a5f52f94a145bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.031788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef892bec8a7455b99a5f52f94a145bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.063234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 2999/5915] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:10 2022] DELETE /identity/v3/users/fef892bec8a7455b99a5f52f94a145bc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:11.064030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:11.068076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.121616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:11.133259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a4596-826f-4b06-ad08-e121679001b0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:11.218657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1183d-db60-4885-a195-8d006aae79e9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:11.231297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:11.233434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:11.234804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:11.235848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] PATH_INFO: `/v3/users/38ddb93a51504e32b2816ee6b6733956` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:11.239793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ddb93a51504e32b2816ee6b6733956)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:11.247100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:11.262850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:11.264202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:11.264661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:11.265226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] PATH_INFO: `/v3/projects/50f7cb0bd6bd410e994ec45db40340ca` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:11.266724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f7cb0bd6bd410e994ec45db40340ca)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:11.269849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:11.348915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ddb93a51504e32b2816ee6b6733956'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.349809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ddb93a51504e32b2816ee6b6733956'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.365915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f7cb0bd6bd410e994ec45db40340ca', 'request_id': 'req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.367199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ddb93a51504e32b2816ee6b6733956'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.373953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f7cb0bd6bd410e994ec45db40340ca', 'request_id': 'req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.385542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f7cb0bd6bd410e994ec45db40340ca', 'request_id': 'req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.393206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ddb93a51504e32b2816ee6b6733956'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.395070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ddb93a51504e32b2816ee6b6733956'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.402100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] The token cache is being invalidate because project 50f7cb0bd6bd410e994ec45db40340ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:11.402953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.403876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1f2d235b-7d71-4476-a1a3-ff0ca9f2dd00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.409094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2917/5916] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:10 2022] DELETE /identity/v3/projects/50f7cb0bd6bd410e994ec45db40340ca => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:11.409094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:11.409094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:11.410105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:11.418439 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3000/5917] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:11 2022] DELETE /identity/v3/users/38ddb93a51504e32b2816ee6b6733956 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:11.419563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:11.423404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.452089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:11.453413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.495714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:11.555147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0c976-665d-4fff-a0be-6bcc3235942f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:11.580335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2918/5918] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:11.581692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:11.594272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:11.598936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:11.599400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:11.599873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] PATH_INFO: `/v3/users/881b1fad2940439988bd4d9c7fa2301b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:11.601600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.602390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] RBAC: Authorizing `identity:delete_user(user_id=881b1fad2940439988bd4d9c7fa2301b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:11.604968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:11.709635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:11.735913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881b1fad2940439988bd4d9c7fa2301b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.738200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881b1fad2940439988bd4d9c7fa2301b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.759357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144c5ed-af9e-4c53-8493-6dd3ded3aa38 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:11.759494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881b1fad2940439988bd4d9c7fa2301b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.764837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:11.765595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:11.765873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:11.766140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:11.773384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.781464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881b1fad2940439988bd4d9c7fa2301b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.782350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2f90b942-ac14-43c2-b9eb-f225e1d1070c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881b1fad2940439988bd4d9c7fa2301b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:11.810120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:11.811980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:11.828103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2919/5919] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:12:11.829047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:11.833720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:11.846526 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3001/5920] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:11 2022] DELETE /identity/v3/users/881b1fad2940439988bd4d9c7fa2301b => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:11.884287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffead41-b272-4f66-835b-101d61506b72 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:12.003881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:12.005228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.005836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.006336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] PATH_INFO: `/v3/projects/7d55af194b014318a32f36f36966b2c9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.008028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d55af194b014318a32f36f36966b2c9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:12.011507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:12.131193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d55af194b014318a32f36f36966b2c9', 'request_id': 'req-4b936e0e-4282-45f1-a202-acf1076a4cb4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.134384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:12.140236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d55af194b014318a32f36f36966b2c9', 'request_id': 'req-4b936e0e-4282-45f1-a202-acf1076a4cb4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.160847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d55af194b014318a32f36f36966b2c9', 'request_id': 'req-4b936e0e-4282-45f1-a202-acf1076a4cb4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.185594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] The token cache is being invalidate because project 7d55af194b014318a32f36f36966b2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:12.187717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.189267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b936e0e-4282-45f1-a202-acf1076a4cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.195020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2920/5921] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:11 2022] DELETE /identity/v3/projects/7d55af194b014318a32f36f36966b2c9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:12.204559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.207314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.208459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.273406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:12.275755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.370703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592db70-35c9-4a8a-bb41-326269c69b18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.371214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592db70-35c9-4a8a-bb41-326269c69b18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.371340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592db70-35c9-4a8a-bb41-326269c69b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.421046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0592db70-35c9-4a8a-bb41-326269c69b18 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:12.434655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0592db70-35c9-4a8a-bb41-326269c69b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.513814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3002/5922] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:12.519512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:12.523794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.558035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40198d50-900b-48c4-9576-90c639c450f9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:12.589949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:12.591971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.592332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.593271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] PATH_INFO: `/v3/projects/0ad8713cc3a0464699bc316e93f94d28` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.600475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad8713cc3a0464699bc316e93f94d28)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:12.604480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:12.641297 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2921/5923] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:12.642683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.643019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.643406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.713200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:12.720169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.762864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad8713cc3a0464699bc316e93f94d28', 'request_id': 'req-a56b782d-1fdc-4773-a09a-76c219d3e2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.777140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad8713cc3a0464699bc316e93f94d28', 'request_id': 'req-a56b782d-1fdc-4773-a09a-76c219d3e2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.798374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad8713cc3a0464699bc316e93f94d28', 'request_id': 'req-a56b782d-1fdc-4773-a09a-76c219d3e2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.817755 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2922/5924] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:12.818776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:12.820518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.833344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] The token cache is being invalidate because project 0ad8713cc3a0464699bc316e93f94d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:12.833700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.834412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e4e19-5cbb-4066-bf5a-265ae17a0660 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:12.835037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.839580 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3003/5925] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:12 2022] DELETE /identity/v3/projects/0ad8713cc3a0464699bc316e93f94d28 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:12.839890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:12.840412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:12.840542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:12.840869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:12.841146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] PATH_INFO: `/v3/projects/09a9f7bf1fc246eda820151e97653b2c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:12.842418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a9f7bf1fc246eda820151e97653b2c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:12.843908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:12.844315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:12.957156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a9f7bf1fc246eda820151e97653b2c', 'request_id': 'req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.967413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a9f7bf1fc246eda820151e97653b2c', 'request_id': 'req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:12.984289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:12.986209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a9f7bf1fc246eda820151e97653b2c', 'request_id': 'req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.037704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] The token cache is being invalidate because project 09a9f7bf1fc246eda820151e97653b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:13.038986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.041005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.046894 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2923/5926] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:12 2022] DELETE /identity/v3/projects/09a9f7bf1fc246eda820151e97653b2c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:13.047852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.054630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.109351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b782d-1fdc-4773-a09a-76c219d3e2e8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.111063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:13.270449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea450e3-d1c5-4b6e-bed9-968fa5838e7b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.283191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.283817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.284051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.284240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.284551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.284997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.285376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.285750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.286672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:13.286838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.287043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.290832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60fe2f-2162-4f21-8429-1032feac880a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.319397 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2924/5927] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:13.320936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121493-b116-4041-9bbe-7a3efa39f295 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.325268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121493-b116-4041-9bbe-7a3efa39f295 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.325666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7121493-b116-4041-9bbe-7a3efa39f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.369617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c7121493-b116-4041-9bbe-7a3efa39f295 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:13.372031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7121493-b116-4041-9bbe-7a3efa39f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.398565 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2925/5928] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:13.399933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.400220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.400494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.416650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.419053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.431548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3004/5929] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:13.432593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.435582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.441289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:13.442935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.464019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f825f-3703-4429-bb01-7f2d970cf60c service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.465183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2926/5930] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:13.470661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.471407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.471946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.472330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] PATH_INFO: `/v3/projects/74780edf6b114f7ca7a9020c041482c3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.472540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.474087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74780edf6b114f7ca7a9020c041482c3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.476590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.492170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad000e54-5127-42c6-a195-6c0e059a0a24 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.497758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.498563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.498852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.499128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.499790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.502017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd480cfa-fbc4-4caa-b0ef-9aee0cf18574 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.550345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2927/5931] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:13.552530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.552819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.553160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.583163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:13.584569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.603903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74780edf6b114f7ca7a9020c041482c3', 'request_id': 'req-f1bf6c78-9502-4a19-a92c-38d855290c8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.619210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74780edf6b114f7ca7a9020c041482c3', 'request_id': 'req-f1bf6c78-9502-4a19-a92c-38d855290c8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.641080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74780edf6b114f7ca7a9020c041482c3', 'request_id': 'req-f1bf6c78-9502-4a19-a92c-38d855290c8e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.668948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] The token cache is being invalidate because project 74780edf6b114f7ca7a9020c041482c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:13.669391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.670154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f1bf6c78-9502-4a19-a92c-38d855290c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:13.678149 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3005/5932] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:13 2022] DELETE /identity/v3/projects/74780edf6b114f7ca7a9020c041482c3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:13.680820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf3c3-a049-44c2-a95e-9ebe8521bfc3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.681498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf3c3-a049-44c2-a95e-9ebe8521bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.681881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acf3c3-a049-44c2-a95e-9ebe8521bfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.696755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:13.728404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-21acf3c3-a049-44c2-a95e-9ebe8521bfc3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:13.730268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21acf3c3-a049-44c2-a95e-9ebe8521bfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.731881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2928/5933] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:13.732755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.735536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.759618 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3006/5934] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:13.763787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.764222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.764846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.772711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336391a2-a632-430f-8da5-ba4ce5ad061f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.785689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.786535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.786802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.787075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.787728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.792611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.817841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:13.828745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.840232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2929/5935] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:13.842840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.864694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d3673-2f75-4611-86ee-510b685d3547 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.865171 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3007/5936] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:13.866211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.871063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:13.884670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.885184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.885466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.885740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.886398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:13.886725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.889113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.915661 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2930/5937] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:13.916626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.928558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226cfc8-b78f-41a1-8881-2416ffb4c405 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.933901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.934843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.935341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.935799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.936607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:13.937083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.939100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.954484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2931/5938] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:13.955992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:13.968634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45f686-398d-4442-b80d-a701ef092688 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:13.975903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:13.976748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.977192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.977626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:13.978429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:13.980931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2804da-99cf-47a2-8d8b-e488e9653b43 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:13.994953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2932/5939] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:13.996398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:13.996690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:13.996981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.010034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cdcd1-c08a-44a1-961f-5efa42d7d7ed None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.017661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.019563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.020372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.021682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.023692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:14.031331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:14.032582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:14.042793 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2933/5940] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:14.046365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.076172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7561560-0639-4d73-88f6-c8d4f1ff151f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.092558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.092558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.096116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.096412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.099422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.101387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.115211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:14.165912 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2934/5941] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:14.168164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.202864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.204594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.252627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3008/5942] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:14.254827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.285965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658180-1556-4fa2-bac2-599cac356722 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.333285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc05a-7078-4877-b487-d4cdbfde9cb5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.338635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.339414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.339705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.339980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.340611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.342695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6a091-4d09-41b9-ba7d-fc0615bff463 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.386486 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2935/5943] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:14.387981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.388281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.388560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.424297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:14.426608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:14.430430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.431090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.431409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.432078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.432857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.435471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.452734 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3009/5944] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:14.453554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.470692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a7530-acc1-45d2-be64-75740daedf4d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.491131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:14.512311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2936/5945] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:14.513427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.668003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.668911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.669199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.670056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.671034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:14.671516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.673476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.681567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07100e4-6443-4ca3-89ff-58c785b579f0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.693982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.695841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.696130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.697579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.699739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.703004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.708953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3010/5946] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:14.710100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.733251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21e66-3af7-4898-a8c5-c91881aa286d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.751300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.751930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.752198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.753242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.754705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.761761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2937/5947] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:14.762891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.768312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.776071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:14.790757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d07b2d-3e81-4559-a1b0-0046f1b54677 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:14.801748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:14.802447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:14.802756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:14.803029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] PATH_INFO: `/v3/projects/34187cd872e346cbb8c818b32fa72b34/users/7a6fcbf3208747959c57681abb011b05/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:14.834301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34187cd872e346cbb8c818b32fa72b34, user_id=7a6fcbf3208747959c57681abb011b05, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:14.837210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:14.842107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3011/5948] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:14.842881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:14.854025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:14.871675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2938/5949] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:14 2022] PUT /identity/v3/projects/34187cd872e346cbb8c818b32fa72b34/users/7a6fcbf3208747959c57681abb011b05/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:14.882063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.006147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9e185-1843-42fd-9680-3c2c7ff02462 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.014967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.016023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.016589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.017572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.019294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.023369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.032203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc2d2d-7e0e-4932-948b-c37bf4e2f783 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.076365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2939/5950] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:14 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:15.081918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.180466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.181403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.181851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.182301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.189139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f624d-33df-4b41-b5d1-2a6813014ad0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.196807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.197490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.197824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.198102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.198807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.200898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.227006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2940/5951] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.228116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.276195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:15.281477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2a943-bbee-4928-8969-72e204c2d13f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.291738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.292442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.292880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.293460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.294396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.299127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8dc87-d4a5-41db-ba61-8b7b8129da0a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.328007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.330389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.335270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2941/5952] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.336782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.337072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.337342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.355103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3012/5953] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:15.363432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.403891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8b7e8-ea44-41de-bad4-0591f41b1103 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.407437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] MFA Rules not processed for user `7a6fcbf3208747959c57681abb011b05`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:15.408798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:15.419647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.420295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.420583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.420857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.421501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.423510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.497123 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3013/5954] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:15 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:15.498068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.541325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2184c2c-a0a6-4c10-9d33-7cf04039cee9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.546379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:15.553179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.553817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.554050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.554326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.555002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.556952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.590942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3014/5955] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.592239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.603643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2942/5956] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:15.608585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.621035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cc802-c282-42e4-945a-14705fd3dbe5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.626015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.626656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.626937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.627212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.628440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.630183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.639901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7568ee-e347-4ca7-a4e3-5efc7ed50040 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.651573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.652320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.652755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.653303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.654176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.657919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.662447 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3015/5957] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.666478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.675848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2943/5958] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.677228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.689772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fae04-6688-4fe2-820f-d08d4e9b8dcf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.690826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11d3b0-6bda-4c51-be56-40f33b73d197 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.697208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.697959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.698348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.698778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.699519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.701444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.701678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.702242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.702405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.702717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] PATH_INFO: `/v3/projects/91767e034252401e9050daccca409c3b/users/d8b9cf454b2748e4928f7609580e1944/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.736701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91767e034252401e9050daccca409c3b, user_id=d8b9cf454b2748e4928f7609580e1944, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.741355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.762113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2944/5959] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:15 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:15.763523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.786977 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3016/5960] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:15 2022] PUT /identity/v3/projects/91767e034252401e9050daccca409c3b/users/d8b9cf454b2748e4928f7609580e1944/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:15.788130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.886091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f0ebf-c047-4cad-937e-bfb04802ae2e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.923140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c1fd-34a6-431f-87b7-80c1455321f9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:15.935099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:15.936982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:15.938486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:15.939940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:15.941490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:15.953205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:15.979187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3017/5961] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:15.980278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:15.998455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b844630-9a82-4228-9dcc-61f919494fbc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:16.007330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:16.009957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.010774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.013251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.014258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:16.014450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:16.015446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.016004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.016484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] PATH_INFO: `/v3/projects/0ac2612454b748afa764d38233fec09d/users/49a978fade24419e8c10c78129cf16ab/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.017666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:16.039186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac2612454b748afa764d38233fec09d, user_id=49a978fade24419e8c10c78129cf16ab, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:16.042909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd4e3-33a9-4450-a3a2-03bcf95ff560 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:16.078570 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2945/5962] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:15 2022] PUT /identity/v3/projects/0ac2612454b748afa764d38233fec09d/users/49a978fade24419e8c10c78129cf16ab/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:16.080485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c53a65-11bc-49fa-942a-32392cffbc7e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.080485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c53a65-11bc-49fa-942a-32392cffbc7e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.081532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c53a65-11bc-49fa-942a-32392cffbc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.091405 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3018/5963] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:15 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:16.094038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:16.127641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e02a9-351d-4a37-814e-3c77a4f4096c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:16.154134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-33c53a65-11bc-49fa-942a-32392cffbc7e None None] MFA Rules not processed for user `d8b9cf454b2748e4928f7609580e1944`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:16.158133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-33c53a65-11bc-49fa-942a-32392cffbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:16.219353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:16.281900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:16.318588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:16.319388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.319728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.320068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] PATH_INFO: `/v3/projects/6d5a86475adf476db71c7b6fb6fe70f2/users/3b2595fa2c3a44329552f6f43af0a7bb/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.346363 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2946/5964] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:16.352124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.352425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.352707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.357806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5a86475adf476db71c7b6fb6fe70f2, user_id=3b2595fa2c3a44329552f6f43af0a7bb, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:16.365024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f96e7b-853a-4737-b925-36d78fe9b55c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:16.396363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:16.406476 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3019/5965] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:16 2022] PUT /identity/v3/projects/6d5a86475adf476db71c7b6fb6fe70f2/users/3b2595fa2c3a44329552f6f43af0a7bb/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:16.406756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:16.406858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.406858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.406858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.478484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] MFA Rules not processed for user `49a978fade24419e8c10c78129cf16ab`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:16.479754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:16.556697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2947/5966] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:16.558292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:16.763870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3de3be-44b6-48ea-b224-9da8db817c4b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:16.769880 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3020/5967] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:16.771060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:16.783216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:16.785356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.785833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.786263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.787615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:16.789630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9809526-8eb9-4087-90e9-a6011d502d7f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:16.810034 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2948/5968] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:16.811548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.811842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.812262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.863527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] MFA Rules not processed for user `3b2595fa2c3a44329552f6f43af0a7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:16.864959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:16.978338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8986df-99d6-498a-89b4-293ed87a0e1d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:16.991266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8b9cf454b2748e4928f7609580e1944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91767e034252401e9050daccca409c3b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:16.994391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:16.994952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:16.995459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] PATH_INFO: `/v3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:16.998441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3021/5969] 10.222.0.52 () {58 vars in 1104 bytes} [Sat Sep 24 21:12:16 2022] GET /identity/v3 => generated 255 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:17.003417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.071909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:17.079161 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2949/5970] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:17.080346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.100681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775701-287c-4378-8b72-e64f09ff25b5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.112636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.115683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.116035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.116555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.117548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:17.126650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:17.189935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a557a3b-04d4-4fa2-a518-94ca2025b268 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.200355 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2950/5971] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:17 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:17.202048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.207563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.208101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.208405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.208686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.237582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:17.239253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:17.250305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:17.290100 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3022/5972] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:17.294717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.356364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478d3af-bd50-491a-b9fe-7c4bfbd58b0d service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.361490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f536e9-833b-4295-900d-7f6a27c7869d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.376981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.378319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.378766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.379179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] PATH_INFO: `/v3/projects/57580819362741b182ab7616cb8793ab/users/b469649953a24842839d689b1f3885be/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.418070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=57580819362741b182ab7616cb8793ab, user_id=b469649953a24842839d689b1f3885be, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:17.422987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:17.463410 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2951/5973] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:17 2022] PUT /identity/v3/projects/57580819362741b182ab7616cb8793ab/users/b469649953a24842839d689b1f3885be/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:17.464419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.572602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8b9cf454b2748e4928f7609580e1944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91767e034252401e9050daccca409c3b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.575290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.575724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.576149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] PATH_INFO: `/v3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.581331 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3023/5974] 10.222.0.52 () {58 vars in 1104 bytes} [Sat Sep 24 21:12:17 2022] GET /identity/v3 => generated 255 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:17.583415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.624827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073be7-26cb-43ee-8d4d-216f9615a669 tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.707845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d530a6c-0037-4467-b9c2-8c24330757ce admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:17.732446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.735786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.737845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.740941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.741759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:17.748735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:17.870840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:17.872212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:17.872501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:17.872782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:17.879974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:17.886800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2952/5975] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:17 2022] POST /identity/v3/projects => generated 383 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:17.890373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:17.893823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:17.967111 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3024/5976] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:17 2022] POST /identity/v3/users => generated 288 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:17.970788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.062167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:18.134365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4de5-9a02-416e-a2f8-8145948f30b4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.167249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.171347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.172198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.176743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.177558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.202981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e6c2-3c29-4768-8cff-f472db3db101 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.203206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.233488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2953/5977] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:18.235141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.251716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fda0e-209b-4a91-aa45-6662768dd790 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.260700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.261332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.261616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.261883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.267073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.269659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.329057 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2954/5978] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:18 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:18.332425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.356545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71e03-9005-434b-9c8e-097ce2af1f12 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.371498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.372149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.373100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.373394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.375437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:18.375578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.391316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.399252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940a174-23c1-4b82-9ba2-b9dca21d834b tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8b9cf454b2748e4928f7609580e1944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '91767e034252401e9050daccca409c3b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.399880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940a174-23c1-4b82-9ba2-b9dca21d834b tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.400165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940a174-23c1-4b82-9ba2-b9dca21d834b tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.400441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940a174-23c1-4b82-9ba2-b9dca21d834b tempest-TestApiDiscovery-1657884562 tempest-TestApiDiscovery-1657884562-project] PATH_INFO: `/v3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.403588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3025/5979] 10.222.0.52 () {58 vars in 1104 bytes} [Sat Sep 24 21:12:17 2022] GET /identity/v3 => generated 255 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:18.405694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf856a4-2f20-4af7-9a32-32a6a5ce88cd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.413454 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2955/5980] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:18.413454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.413879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf856a4-2f20-4af7-9a32-32a6a5ce88cd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.413981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf856a4-2f20-4af7-9a32-32a6a5ce88cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.452263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176efe7-c3aa-4595-8ef9-7dd9b93e1028 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.471970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fbf856a4-2f20-4af7-9a32-32a6a5ce88cd None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:18.473243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fbf856a4-2f20-4af7-9a32-32a6a5ce88cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:18.477849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.478569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.478856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.479136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] PATH_INFO: `/v3/projects/57580819362741b182ab7616cb8793ab/users/b469649953a24842839d689b1f3885be/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.488683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=57580819362741b182ab7616cb8793ab, user_id=b469649953a24842839d689b1f3885be, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.491582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.524813 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2956/5981] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:18 2022] PUT /identity/v3/projects/57580819362741b182ab7616cb8793ab/users/b469649953a24842839d689b1f3885be/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:18.525583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.667544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3026/5982] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:18.671389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877effe-2abb-48eb-8a09-d2066b894bac None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.671689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877effe-2abb-48eb-8a09-d2066b894bac None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.671961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877effe-2abb-48eb-8a09-d2066b894bac None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.681192 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3027/5983] 10.222.0.52 () {56 vars in 895 bytes} [Sat Sep 24 21:12:18 2022] GET /identity/ => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:12:18.683570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877effe-2abb-48eb-8a09-d2066b894bac None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.708536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200d27-d08c-4c8a-a568-3a3ab7e1a9fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.714212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.715757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.716043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.716498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.717535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.718345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877effe-2abb-48eb-8a09-d2066b894bac None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:18.735792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:18.736478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.736758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.736886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ff7b5-264e-4184-a8fe-a3c13fa1b5c9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.737649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] PATH_INFO: `/v3/projects/95e2bd9b02004a22a2720cc852d7d46d/users/24c5a6a27f044d3aa2207086db72cb6a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.764015 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2957/5984] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:18.764015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.764015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.764015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.781776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e2bd9b02004a22a2720cc852d7d46d, user_id=24c5a6a27f044d3aa2207086db72cb6a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:18.789847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bc0ab-71c7-4844-a0e9-102b4bcf439e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:18.812465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] MFA Rules not processed for user `b469649953a24842839d689b1f3885be`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:18.814077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:18.818769 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3028/5985] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:18 2022] PUT /identity/v3/projects/95e2bd9b02004a22a2720cc852d7d46d/users/24c5a6a27f044d3aa2207086db72cb6a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:18.822054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cb8f-347a-4174-be4d-bd72750c294e None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:18.822381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cb8f-347a-4174-be4d-bd72750c294e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:18.824295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cb8f-347a-4174-be4d-bd72750c294e None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:18.828851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3029/5986] 10.222.0.52 () {56 vars in 895 bytes} [Sat Sep 24 21:12:18 2022] GET /identity/ => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:12:18.834167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1cb8f-347a-4174-be4d-bd72750c294e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.938000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2958/5987] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:18.938952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:18.975851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:19.073943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1cb8f-347a-4174-be4d-bd72750c294e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.082406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.083272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.083627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.083970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.108596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e634c1b-2081-4703-8e4b-41a58a5453d1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.122125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.123109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.125471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.126075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] PATH_INFO: `/v3/projects/a0d8ebc5380b4293baac1d506103eb4c/users/331e5acba3814c09bc74dd18b0601072/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.170780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d8ebc5380b4293baac1d506103eb4c, user_id=331e5acba3814c09bc74dd18b0601072, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.173162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.175828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.180309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.186921 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3030/5988] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:19.187923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:19.206190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05fdd6-58d1-4cae-8426-bb7686a75760 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.218600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.220099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.220478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.224464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.226488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.230609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.246157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3031/5989] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:19.247268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:19.254223 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2959/5990] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:18 2022] PUT /identity/v3/projects/a0d8ebc5380b4293baac1d506103eb4c/users/331e5acba3814c09bc74dd18b0601072/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:19.258342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:19.258923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:19.264199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:19.267366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601879e7-1bb8-4dbe-9188-876e3e820a85 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.360533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.361459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.361937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.362424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] PATH_INFO: `/v3/users/d8b9cf454b2748e4928f7609580e1944` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.364344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b9cf454b2748e4928f7609580e1944)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.373796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.487580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b9cf454b2748e4928f7609580e1944'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:19.489358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b9cf454b2748e4928f7609580e1944'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:19.505187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b9cf454b2748e4928f7609580e1944'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:19.526773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b9cf454b2748e4928f7609580e1944'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:19.528621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b9cf454b2748e4928f7609580e1944'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:19.532897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc86d7e-76a3-4db2-8089-b253709216d1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.548962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.550399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.550776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.551119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.557763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:19.573342 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3032/5991] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:19 2022] DELETE /identity/v3/users/d8b9cf454b2748e4928f7609580e1944 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:19.574427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:19.579381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:19.764026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.766143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.788119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b731dab-350c-4806-8834-2851ea31ae5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.797591 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2960/5992] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:19.799356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:19.800243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.801920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.802805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:19.803167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.804920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] PATH_INFO: `/v3/projects/95e2bd9b02004a22a2720cc852d7d46d/users/24c5a6a27f044d3aa2207086db72cb6a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.809843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e2bd9b02004a22a2720cc852d7d46d, user_id=24c5a6a27f044d3aa2207086db72cb6a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.820012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672619f4-b2d6-491f-8898-35139cfc9bc9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.842749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5ff74-3812-42d7-aea8-8b478577662b service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:19.850179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3033/5993] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:19 2022] PUT /identity/v3/projects/95e2bd9b02004a22a2720cc852d7d46d/users/24c5a6a27f044d3aa2207086db72cb6a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:19.853063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24750b80-648b-4518-b6de-00c871d2826d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.854020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24750b80-648b-4518-b6de-00c871d2826d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.854354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24750b80-648b-4518-b6de-00c871d2826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.919588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-24750b80-648b-4518-b6de-00c871d2826d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:19.922302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-24750b80-648b-4518-b6de-00c871d2826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:19.923632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:19.941701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:19.942709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.942998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.943278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:19.945868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:19.946578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2a6ac-542e-4511-8e26-db763da97676 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:19.958994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:19.976258 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2961/5994] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:19.981653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:19.982133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:19.982546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:20.057854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] MFA Rules not processed for user `24c5a6a27f044d3aa2207086db72cb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:20.061221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:20.103223 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3034/5995] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:20.109882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:20.114263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:20.115562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:20.152709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:20.154400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:20.256843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2962/5996] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:20.257954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:20.281448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3035/5997] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:12:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:20.282402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:20.286443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:20.292019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8af5bb-ef68-4041-9d66-dbf05f119a5b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:20.306413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:20.307580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:20.307982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:20.308274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] PATH_INFO: `/v3/projects/a0d8ebc5380b4293baac1d506103eb4c/users/331e5acba3814c09bc74dd18b0601072/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:20.312069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d8ebc5380b4293baac1d506103eb4c, user_id=331e5acba3814c09bc74dd18b0601072, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:20.317082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:20.336841 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2963/5998] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:20 2022] PUT /identity/v3/projects/a0d8ebc5380b4293baac1d506103eb4c/users/331e5acba3814c09bc74dd18b0601072/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:20.337895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:20.349494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:20.500152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e3e63-00c1-4a14-8cd6-8267f09a5ada None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:20.508143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762888c2-b6bb-4e04-8eb7-e8b0956c95bd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:20.534404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:20.535555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:20.535555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:20.535789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:20.702790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:20.706578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:20.706874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:20.707585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:20.745264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:20.751574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:20.792142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2964/5999] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:20.797140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:20.802231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:20.811133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:20.907204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:20.923553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:20.927328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c493-3ff0-42c4-a7a6-be1e1ace3ac1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:20.952835 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3036/6000] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:20.957750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:20.958056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:20.958441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:21.044291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] MFA Rules not processed for user `331e5acba3814c09bc74dd18b0601072`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:21.045736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:21.152341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569c94-f493-4a4a-a831-cf7e96497da7 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:21.164182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:21.165909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:21.166339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:21.166763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:21.366125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:21.372180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8980cc1-a383-40a6-a9bf-c0c4133ec9aa service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:21.403825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2965/6001] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:21.406012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:21.406817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:21.407485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:21.482294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3037/6002] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:21.504371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:21.545719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] MFA Rules not processed for user `57c163c13444454c9a23703c08a9cc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:21.547620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:21.552998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e0e14-3064-4f4c-bd94-2a6736bf2a6f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:21.566941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:21.570740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:21.573559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:21.575767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:21.614425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:21.620222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:21.694746 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3038/6003] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:21.821627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2966/6004] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:21.834732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:21.837711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:22.042801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc79da5-4da8-42ad-aa4a-478e208458f2 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:22.052651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:22.054549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:22.055197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:22.055702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:22.074733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:22.088604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:22.111654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3039/6005] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:22.220679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:22.248536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a5387-992b-4b1c-8ea2-390c3a2dc325 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:22.258248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:22.259033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:22.261481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:22.261481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:22.264498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:22.366289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:22.421998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:22.423987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:22.453582 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2967/6006] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:23.095502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:23.098074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.131817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5824c-c2ba-42e9-8651-af32be956502 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:23.140171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:23.140893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.141305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.141742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] PATH_INFO: `/v3/projects/91767e034252401e9050daccca409c3b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.161309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91767e034252401e9050daccca409c3b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:23.170795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:23.212862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:23.227687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd941316-ff3f-484d-9813-bc1ed939e8f2 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:23.243832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:23.244851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.245496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.245978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] PATH_INFO: `/v3/users/3b2595fa2c3a44329552f6f43af0a7bb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.251047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2595fa2c3a44329552f6f43af0a7bb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:23.253572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:23.269216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:23.323374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91767e034252401e9050daccca409c3b', 'request_id': 'req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.335526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91767e034252401e9050daccca409c3b', 'request_id': 'req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.358790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91767e034252401e9050daccca409c3b', 'request_id': 'req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.378476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2595fa2c3a44329552f6f43af0a7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.379650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2595fa2c3a44329552f6f43af0a7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.394752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2595fa2c3a44329552f6f43af0a7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.397515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] The token cache is being invalidate because project 91767e034252401e9050daccca409c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:23.398015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.403102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-868601a9-d11d-4df1-bc16-f44c4b6ac9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.405330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3040/6007] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:23 2022] DELETE /identity/v3/projects/91767e034252401e9050daccca409c3b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:23.405798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2595fa2c3a44329552f6f43af0a7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.407181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7af52c6-fb65-4e4a-a25f-cd189f2a3c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2595fa2c3a44329552f6f43af0a7bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:23.407360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a029b2-9b10-41b9-bfe6-c6ceabe7b127 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.407917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a029b2-9b10-41b9-bfe6-c6ceabe7b127 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.408476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a029b2-9b10-41b9-bfe6-c6ceabe7b127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.455400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e7a029b2-9b10-41b9-bfe6-c6ceabe7b127 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:23.459965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e7a029b2-9b10-41b9-bfe6-c6ceabe7b127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.477625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2968/6008] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:23 2022] DELETE /identity/v3/users/3b2595fa2c3a44329552f6f43af0a7bb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:23.479308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.479904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.481296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.526423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:23.528020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.574515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3041/6009] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:23.576002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a809b3-a6fc-4e30-a645-cb0bde8be394 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.576496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a809b3-a6fc-4e30-a645-cb0bde8be394 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.576864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a809b3-a6fc-4e30-a645-cb0bde8be394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.619928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d5a809b3-a6fc-4e30-a645-cb0bde8be394 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:23.619928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5a809b3-a6fc-4e30-a645-cb0bde8be394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.697283 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2969/6010] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:23.698478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:23.700220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.791953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3042/6011] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:23.805622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d904-200d-4174-a427-7389bad8226b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.806633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d904-200d-4174-a427-7389bad8226b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.807909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35d904-200d-4174-a427-7389bad8226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.836072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:23.878143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ca35d904-200d-4174-a427-7389bad8226b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:23.879620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c8ef0-3128-4874-9e0f-365428f01b1f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:23.881608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca35d904-200d-4174-a427-7389bad8226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:23.891357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:23.892045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:23.892429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:23.892980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:23.915051 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3043/6012] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:23.917330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35d904-200d-4174-a427-7389bad8226b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:23.934874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca35d904-200d-4174-a427-7389bad8226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:24.053496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:24.057589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.063157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.063380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35d904-200d-4174-a427-7389bad8226b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.069476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.070850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.071376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.072045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.088591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.090985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.095889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2970/6013] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:24.097538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.107413 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3044/6014] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:24.122770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5520ff-6341-44a7-9888-0a96f6afa9d5 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.137852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.138534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.138817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.139115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.140014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:24.140128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.146218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.178100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2971/6015] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:24.182766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.195279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c871a40-e7c9-43c7-b3de-c341e3092c31 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.229163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.234204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.237247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.237643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.238244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.240208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747d10c-3b09-48e9-a07a-0ddd5db064dd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.325605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3045/6016] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:24 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:24.328567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.345052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cdb16-de52-4dc8-be87-aa9f6fec0480 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.468919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.469560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.469895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.505824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.506704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.507226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.507508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.508156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.510124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.513453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:24.514774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:24.555001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2972/6017] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:24 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:24.559627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.585530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c16f9-ac23-41c1-91eb-ef1eb7d810f5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.595602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.596384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.597704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.598104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.598851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.601537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.611660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3046/6018] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:12:24.645552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.664058 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2973/6019] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:12:24.665114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.691088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad8513-ea7f-4c02-b940-24c536141bcc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.703710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.704340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.704615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.706901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.707637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.712695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.734189 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2974/6020] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:24.739136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.766108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53719d7a-ce10-4d31-a897-375847da720a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.777463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.778389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.778998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.779437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] PATH_INFO: `/v3/projects/2a5c401b74c340cb800f10da0a5903a5/users/5dd41a2729a14d8d86a5ad55074c58e5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.817089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5c401b74c340cb800f10da0a5903a5, user_id=5dd41a2729a14d8d86a5ad55074c58e5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.819810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.832601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f16d-77e4-44e2-b6a0-257ba52c3891 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.846836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.847537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.847816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.848087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.848723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2975/6021] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:24 2022] PUT /identity/v3/projects/2a5c401b74c340cb800f10da0a5903a5/users/5dd41a2729a14d8d86a5ad55074c58e5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:24.849871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:24.860748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:24.876384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.879010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11201ec-9e48-47f0-9d14-4b4265a86595 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:24.906085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3047/6022] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:24.907860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.908062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.909222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.913790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:24.982090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae36e06-b3ab-45cc-a788-d340a175ca75 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:24.982313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] MFA Rules not processed for user `5dd41a2729a14d8d86a5ad55074c58e5`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:24.983492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:24.989985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:24.990867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:24.991198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:24.991483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] PATH_INFO: `/v3/users/49a978fade24419e8c10c78129cf16ab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:24.995083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49a978fade24419e8c10c78129cf16ab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:24.998887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:25.122447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49a978fade24419e8c10c78129cf16ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.124766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49a978fade24419e8c10c78129cf16ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.156170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49a978fade24419e8c10c78129cf16ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.173868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:25.182385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49a978fade24419e8c10c78129cf16ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.183979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-47913fb2-b4bf-4288-9f4a-65b68e35c128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49a978fade24419e8c10c78129cf16ab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.200273 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3048/6023] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:25.200760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:25.203223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.226202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdaccc-0653-4fb2-8a1f-3df3f94186b1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:25.239292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:25.239609 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2976/6024] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:24 2022] DELETE /identity/v3/users/49a978fade24419e8c10c78129cf16ab => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:25.241509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.241793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.242071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] PATH_INFO: `/v3/projects/6d5a86475adf476db71c7b6fb6fe70f2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.244785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.245494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.245781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.250463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5a86475adf476db71c7b6fb6fe70f2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:25.254374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:25.304440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:25.308127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.379593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5a86475adf476db71c7b6fb6fe70f2', 'request_id': 'req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.399553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5a86475adf476db71c7b6fb6fe70f2', 'request_id': 'req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.413367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5a86475adf476db71c7b6fb6fe70f2', 'request_id': 'req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.414920 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2977/6025] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:25.417168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:25.432606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.447458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] The token cache is being invalidate because project 6d5a86475adf476db71c7b6fb6fe70f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:25.447458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.447458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-adea9bc1-f0a0-4d03-96bc-e03fd4cacf39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:25.449220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3049/6026] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:25 2022] DELETE /identity/v3/projects/6d5a86475adf476db71c7b6fb6fe70f2 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:25.450475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.450783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.451063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.486535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:25.488464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.521957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:25.531261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d8f9-8520-4818-87d3-63fb2efe799a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:25.597556 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3050/6027] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:25.598538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:25.600073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.671993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:25.672618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.672928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.673181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.675475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.710781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9dde6-e8f9-46fb-8a30-bd9098b7284e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:25.716561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:25.717596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.717972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.718347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.720730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.748866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:25.814706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:25.816904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:25.829931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:25.831293 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3051/6028] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:25.831542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:25.832603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:25.835666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.854736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2978/6029] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:25.855614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:25.867212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3708846-e42e-43d5-acdc-b955347af10f service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:25.867423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:25.882229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:25.883196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:25.883645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:25.884106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] PATH_INFO: `/v3/users/331e5acba3814c09bc74dd18b0601072` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:25.887744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=331e5acba3814c09bc74dd18b0601072)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:25.890793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:26.039531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331e5acba3814c09bc74dd18b0601072'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:26.040552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331e5acba3814c09bc74dd18b0601072'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:26.066629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc1983-5c71-4eab-ac14-af81cd30fd2c service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:26.073857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331e5acba3814c09bc74dd18b0601072'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:26.092536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:26.094295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:26.095513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:26.095871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:26.104392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:26.107185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331e5acba3814c09bc74dd18b0601072'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:26.111082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-46e101e4-f174-46dc-ada4-66356a09f499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331e5acba3814c09bc74dd18b0601072'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:26.158007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:26.161571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:26.175425 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3052/6030] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:25 2022] DELETE /identity/v3/users/331e5acba3814c09bc74dd18b0601072 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:26.176750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bab75-905e-4b1c-a695-55b43e2d39b6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:26.177038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bab75-905e-4b1c-a695-55b43e2d39b6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:26.177313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bab75-905e-4b1c-a695-55b43e2d39b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:26.209820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2979/6031] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:26.251677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-134bab75-905e-4b1c-a695-55b43e2d39b6 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:26.252975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-134bab75-905e-4b1c-a695-55b43e2d39b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:26.387986 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3053/6032] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:26.403544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:26.405402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:26.441265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415196b-ab6f-4a0c-be8b-f5b625c82463 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:26.601769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:26.619411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:26.622850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:26.623407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:26.623927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:26.649253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:26.652232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:26.679854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2980/6033] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:27.015705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.015934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.016210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.076873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:27.077966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:27.109857 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3054/6034] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:27.128725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:27.145846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081ee67-acc3-4363-b45d-ec72acd29b6f service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:27.161362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:27.161985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.162266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.162561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.182394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:27.184057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded7cd9-c742-47f2-aea2-16f0f2489edb service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:27.203280 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2981/6035] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:27.304359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:27.305805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:27.380682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:27.453742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f36e43-0257-4cd9-b43a-e88139b70f64 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:27.465532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:27.466191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.466472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.466772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] PATH_INFO: `/v3/projects/a0d8ebc5380b4293baac1d506103eb4c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.467951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d8ebc5380b4293baac1d506103eb4c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:27.469890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:27.532355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007127fe-5ea5-4df2-8c4d-230cebf5082a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.533496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007127fe-5ea5-4df2-8c4d-230cebf5082a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.534681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007127fe-5ea5-4df2-8c4d-230cebf5082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.605569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-007127fe-5ea5-4df2-8c4d-230cebf5082a None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:27.609186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-007127fe-5ea5-4df2-8c4d-230cebf5082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:27.631998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d8ebc5380b4293baac1d506103eb4c', 'request_id': 'req-c966546e-a6fe-4028-a20b-f51a91d39b19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:27.655046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d8ebc5380b4293baac1d506103eb4c', 'request_id': 'req-c966546e-a6fe-4028-a20b-f51a91d39b19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:27.679619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d8ebc5380b4293baac1d506103eb4c', 'request_id': 'req-c966546e-a6fe-4028-a20b-f51a91d39b19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:27.703242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] The token cache is being invalidate because project a0d8ebc5380b4293baac1d506103eb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:27.703814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:27.704966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c966546e-a6fe-4028-a20b-f51a91d39b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:27.714195 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3055/6036] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:27 2022] DELETE /identity/v3/projects/a0d8ebc5380b4293baac1d506103eb4c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:27.718015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.718484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.719092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.787771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2982/6037] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 24 21:12:27.789335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:27.791542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:27.791990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:27.793766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] MFA Rules not processed for user `57c163c13444454c9a23703c08a9cc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:27.795117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:27.863416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:27.872315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:27.977765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3056/6038] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:27.981576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:27.991455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.014024 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2983/6039] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:28.014957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:28.022686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.088089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ff516-2ee5-4da5-a4fa-46a111b824c1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:28.100662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:28.101850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.102326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.102824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] PATH_INFO: `/v3/users/5dd41a2729a14d8d86a5ad55074c58e5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.104387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd41a2729a14d8d86a5ad55074c58e5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:28.110219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:28.271519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557761b-fe20-4023-ab38-251990b0c672 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:28.295342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:28.295991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.296272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.298951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.307119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd41a2729a14d8d86a5ad55074c58e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.309584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd41a2729a14d8d86a5ad55074c58e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.312856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:28.324698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:28.327184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:28.353496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd41a2729a14d8d86a5ad55074c58e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.356412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3057/6040] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:12:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:12:28.358841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:28.367316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.382629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd41a2729a14d8d86a5ad55074c58e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.386084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a99e063-2ffc-4360-b49b-20d4cede723d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd41a2729a14d8d86a5ad55074c58e5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.418352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7edf06-5777-4233-8790-99f22e637ccf service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:28.435509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:28.436476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.436940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:28.440062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.440868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] PATH_INFO: `/v3/projects/0ac2612454b748afa764d38233fec09d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.443276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac2612454b748afa764d38233fec09d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:28.450257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2984/6041] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:28 2022] DELETE /identity/v3/users/5dd41a2729a14d8d86a5ad55074c58e5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:28.450837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:28.452189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.452611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.460720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.503420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:28.505853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.626994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac2612454b748afa764d38233fec09d', 'request_id': 'req-326a29f0-f752-446a-a4f3-8cee5379e621'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.657180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac2612454b748afa764d38233fec09d', 'request_id': 'req-326a29f0-f752-446a-a4f3-8cee5379e621'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.698855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac2612454b748afa764d38233fec09d', 'request_id': 'req-326a29f0-f752-446a-a4f3-8cee5379e621'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.737569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] The token cache is being invalidate because project 0ac2612454b748afa764d38233fec09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:28.738427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.739329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-326a29f0-f752-446a-a4f3-8cee5379e621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:28.742611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3058/6042] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:28 2022] DELETE /identity/v3/projects/0ac2612454b748afa764d38233fec09d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:28.747784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.748488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.748991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.781797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:28.784023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.806020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2985/6043] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:28.810089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:28.827421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.867901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5c7f3-aceb-46f5-818f-a7f4963c1542 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:28.962175 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3059/6044] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:28.963385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:28.971831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:28.972096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:28.972973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:28.973442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:28.973882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] PATH_INFO: `/v3/users/73bfde6cb59b4f99be577627805610f6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:28.976031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=73bfde6cb59b4f99be577627805610f6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:28.978339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:28.993111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345d9ab-09d5-4449-b6a4-9b282a8a0db8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.000195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.003797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.004185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.004708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] PATH_INFO: `/v3/users/24c5a6a27f044d3aa2207086db72cb6a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.014759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c5a6a27f044d3aa2207086db72cb6a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.016555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.077959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73bfde6cb59b4f99be577627805610f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.078829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73bfde6cb59b4f99be577627805610f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.108785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73bfde6cb59b4f99be577627805610f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.139277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73bfde6cb59b4f99be577627805610f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.140136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73bfde6cb59b4f99be577627805610f6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.141089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c5a6a27f044d3aa2207086db72cb6a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.141991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c5a6a27f044d3aa2207086db72cb6a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.158123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c5a6a27f044d3aa2207086db72cb6a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.181187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c5a6a27f044d3aa2207086db72cb6a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.182316 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2986/6045] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:28 2022] DELETE /identity/v3/users/73bfde6cb59b4f99be577627805610f6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:29.182929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c5a6a27f044d3aa2207086db72cb6a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:29.183371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.185339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:29.185755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:29.209041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3060/6046] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:28 2022] DELETE /identity/v3/users/24c5a6a27f044d3aa2207086db72cb6a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:29.210092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.211589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:29.229658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:29.340228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00da1d0-573d-4f89-896e-ba47c8152773 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.342159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c756a65-5ce7-47dc-ba99-81de724640dd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.351532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.352162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.352503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.352780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.358235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:29.503259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.504068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.504355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.504647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.505431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:29.505736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.507557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.508673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.510483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc73a0-d946-41da-be4e-bb7643ccbe40 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.523635 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3061/6047] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:29.525274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.525772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.526213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.541743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2987/6048] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:29.543007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.547579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:29.572477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63995026-f960-4590-9559-cd22dff6d16a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.582201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.583279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.583934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.584369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.585174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.587346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.588906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:29.590184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:29.615361 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3062/6049] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:29.648139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2988/6050] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:29 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:29.650104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.656540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.673994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44980fc1-54ad-4ca5-8002-186778909035 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.787075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.788475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.789147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.792915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.823555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f58418-4892-4f2d-8b9a-b3034391debd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.836030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.838126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.840015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.842232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.843586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.850662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.913398 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2989/6051] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:29 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:12:29.918194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:29.927394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:29.943286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e9971-1cc2-47f7-959a-7175fc14ebff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:29.955013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:29.958067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:29.959096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:29.960982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:29.963569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.963890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:29.965065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.974619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:29.985225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3063/6052] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:12:29.996642 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2990/6053] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:29.999590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.014397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531e19f-10a3-4b87-9c5b-a7f6d0e06503 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.024349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.024979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.025258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.025576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.026228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.028029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f091bb-56b6-4238-a6ea-f490f2887b0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.052625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.053810 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3064/6054] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:30.068166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b673-a12b-4ed2-aba5-358822f3b674 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.078943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.079946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.080949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.081393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] PATH_INFO: `/v3/projects/829a61ff30bc493397b34887fdb0e65e/users/e80162fcacaf45668a342480d116fcdc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.108815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:30.118030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=829a61ff30bc493397b34887fdb0e65e, user_id=e80162fcacaf45668a342480d116fcdc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.121112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.153806 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2991/6055] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:30 2022] PUT /identity/v3/projects/829a61ff30bc493397b34887fdb0e65e/users/e80162fcacaf45668a342480d116fcdc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:30.166937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.167301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.167492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.215768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] MFA Rules not processed for user `e80162fcacaf45668a342480d116fcdc`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:30.218193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:30.339918 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3065/6056] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:30.344269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.510848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962dbc9-ba20-4046-94cb-a7e5e8c88bf3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.523124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.523771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.524058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.524340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.524996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.527644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.588273 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2992/6057] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:30 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:30.597114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.614334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd00fc0-aae7-4f28-999a-948876877cf6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.762353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.763030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.763319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.763592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.765893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.768555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.822975 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3066/6058] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:30 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:30.826203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.843995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b446a-7b93-4d75-8ef0-a50ec0b5fb42 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.860804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.860804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.860804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.860804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.861288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.864826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.895760 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2993/6059] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:30.910533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:30.931275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ff4f-de15-4774-ac35-f653b4101a1c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:30.941208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:30.942094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:30.942522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:30.942920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] PATH_INFO: `/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd/users/b3fccd0dc21c46f5a1c7d91d9514e71b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:30.969490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e693ae1e6194ea9a07e54f3beaedbbd, user_id=b3fccd0dc21c46f5a1c7d91d9514e71b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:30.975202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:30.998786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3067/6060] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:30 2022] PUT /identity/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd/users/b3fccd0dc21c46f5a1c7d91d9514e71b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:31.000470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:31.022021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ec2a2-b35d-445f-806e-4b23ac7ff9cf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:31.074551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:31.077545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.188695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:31.189064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:31.189199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.189199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.189407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.190256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:31.192124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:31.209383 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2994/6061] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:31.210238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:31.214076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.249143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d22f0-15f1-4152-9e10-60a63d390d9b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:31.257295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:31.257295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.257732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.258128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] PATH_INFO: `/v3/projects/c6f49a85802b4e3fa4fe8cd939afb037` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.259404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f49a85802b4e3fa4fe8cd939afb037)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:31.260735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0534e14-5221-43da-bec1-5603f3de515b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:31.261926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:31.276583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:31.277452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.277845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.278279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] PATH_INFO: `/v3/projects/95e2bd9b02004a22a2720cc852d7d46d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.280603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e2bd9b02004a22a2720cc852d7d46d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:31.284255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:31.346894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f49a85802b4e3fa4fe8cd939afb037', 'request_id': 'req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.359563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f49a85802b4e3fa4fe8cd939afb037', 'request_id': 'req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.375422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e2bd9b02004a22a2720cc852d7d46d', 'request_id': 'req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.378812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f49a85802b4e3fa4fe8cd939afb037', 'request_id': 'req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.385037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:31.391037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e2bd9b02004a22a2720cc852d7d46d', 'request_id': 'req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.409987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e2bd9b02004a22a2720cc852d7d46d', 'request_id': 'req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.412101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] The token cache is being invalidate because project c6f49a85802b4e3fa4fe8cd939afb037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:31.412962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.417193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.419758 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3068/6062] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:31 2022] DELETE /identity/v3/projects/c6f49a85802b4e3fa4fe8cd939afb037 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:31.421799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:31.429731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.434676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] The token cache is being invalidate because project 95e2bd9b02004a22a2720cc852d7d46d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:31.435096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.436009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ca991c1c-9fbe-41a9-9d70-6ae7b251d438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:31.443444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2995/6063] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:31 2022] DELETE /identity/v3/projects/95e2bd9b02004a22a2720cc852d7d46d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:31.445732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ad4a-5ae2-45cd-ada7-e2a1167435fd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.445732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ad4a-5ae2-45cd-ada7-e2a1167435fd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.445732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ad4a-5ae2-45cd-ada7-e2a1167435fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.475640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8377ad4a-5ae2-45cd-ada7-e2a1167435fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:31.477476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8377ad4a-5ae2-45cd-ada7-e2a1167435fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.541725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996a7d0-800b-41d2-9ad8-abc4f6589e6d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:31.604413 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2996/6064] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:31.606316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.608115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.608652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.647967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:31.649910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.657258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:31.657952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.659596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.659710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] PATH_INFO: `/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd/users/b3fccd0dc21c46f5a1c7d91d9514e71b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.661728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e693ae1e6194ea9a07e54f3beaedbbd, user_id=b3fccd0dc21c46f5a1c7d91d9514e71b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:31.665166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af16f-dafe-4e6a-8843-225f6d844597 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:31.666801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2997/6065] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:31.668518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:31.670453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.682512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:31.687385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2f9f-f0af-4d52-8b19-ed1efb9cec20 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:31.698591 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3069/6066] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:31 2022] PUT /identity/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd/users/b3fccd0dc21c46f5a1c7d91d9514e71b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:31.701708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2a37-c0b4-4ecb-bb02-5a7240768808 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.702865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2a37-c0b4-4ecb-bb02-5a7240768808 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.703698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a2a37-c0b4-4ecb-bb02-5a7240768808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.765436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fe3a2a37-c0b4-4ecb-bb02-5a7240768808 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:31.767419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fe3a2a37-c0b4-4ecb-bb02-5a7240768808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.828836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:31.829777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.830213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.831019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] PATH_INFO: `/v3/projects/2a5c401b74c340cb800f10da0a5903a5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.832557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5c401b74c340cb800f10da0a5903a5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:31.836035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:31.872669 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3070/6067] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:31.872669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cfa7-2cb4-400e-bcc1-274d16a08b61 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.872669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cfa7-2cb4-400e-bcc1-274d16a08b61 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.872669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cfa7-2cb4-400e-bcc1-274d16a08b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.897348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-04b1cfa7-2cb4-400e-bcc1-274d16a08b61 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:31.899127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-04b1cfa7-2cb4-400e-bcc1-274d16a08b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:31.938081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3071/6068] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:31.942380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:31.942906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:31.943340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:31.987285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5c401b74c340cb800f10da0a5903a5', 'request_id': 'req-161b457b-a33a-4deb-8180-b23c645b62af'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:32.001710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] MFA Rules not processed for user `b3fccd0dc21c46f5a1c7d91d9514e71b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:32.004251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5c401b74c340cb800f10da0a5903a5', 'request_id': 'req-161b457b-a33a-4deb-8180-b23c645b62af'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:32.006760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.023881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5c401b74c340cb800f10da0a5903a5', 'request_id': 'req-161b457b-a33a-4deb-8180-b23c645b62af'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:32.047871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] The token cache is being invalidate because project 2a5c401b74c340cb800f10da0a5903a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:32.048344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:32.049651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:32.055668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2998/6069] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:31 2022] DELETE /identity/v3/projects/2a5c401b74c340cb800f10da0a5903a5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:32.056704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.058278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.069397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:32.125931 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3072/6070] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:32.125931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.125931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.196692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b457b-a33a-4deb-8180-b23c645b62af admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.210597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.211368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.211777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.212151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.213091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:32.213500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.215596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ad56d-f04c-4fc7-9fb7-20c00cedd431 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.234766 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 2999/6071] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:32.237185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.237588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.237973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.249681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1263e-d122-491f-bf29-05642838f5c6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.263116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:32.263865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.264491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.264767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.264952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.265073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.265670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:32.265974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.268713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.277101 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3000/6072] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:32.281485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.283993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.286888 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3073/6073] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:32.287876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.315782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f07d0b-598d-44e5-8c28-40966d0987fc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.327025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.327795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.329421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.329922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.331255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.337007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c39339-27b3-4d82-aeb3-8d5b4010ef53 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.358197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:32.383473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815499d6-8225-4074-b1b5-2ab337049d22 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.388754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.389801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.390249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.390751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.391875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3074/6074] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:32.393909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.394278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.394584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.397960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.428286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:32.429563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:32.581033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3075/6075] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:32.582724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.600921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.603732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d2c8a-7b19-4e08-baf7-1f35314524d4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.605608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.620852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.621482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.621765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.622032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.622854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.626161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.630229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3001/6076] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:32.631300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.655769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04083cb-8799-4787-9f7f-56570b5a8d2e service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.670250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.671460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.671901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.672430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.673754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.677654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.690296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3076/6077] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:32.693792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.719461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3002/6078] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:32.720411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.756864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c32e3-a011-41fa-9ca9-7cda270d229f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.863794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:32.909055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c7ce2-7170-4520-9dfc-5f28c7838983 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:32.924914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.925613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:32.925775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.926891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:32.927175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.927447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.928115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:32.928445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.930053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.930589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:32.931049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:32.932054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:32.934628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:32.971599 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3077/6079] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:32.972884 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3003/6080] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:32.977638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.983536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:32.999160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9bfa-b44d-437b-b6d4-c3240e05adb0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.004310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f378-7d70-4f21-8827-3ff6c01f9a62 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.012014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.012858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.013315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.013673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.014461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.016838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.017475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.018420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.018726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.018982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.019371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.020973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.057243 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3078/6081] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.058643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.100852 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3004/6082] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:32 2022] POST /identity/v3/projects => generated 361 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:33.101780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.106560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1985b7c-5152-4601-8049-9ff4b6b7af13 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.270851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:33.300183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.300823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.301098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.301370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.317156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.318966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.321121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdfe59-b120-49d1-935b-6ba4784b1cec admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.339915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.340563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.340846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.341180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.341851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.343690 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3079/6083] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.343944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.345420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.361238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549586a1-5f1a-4ffa-9c91-15017cae648d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.367742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.368428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.368711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.368984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.369678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.372653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.378231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3005/6084] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.378856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.396856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b180-6d92-4c56-bbdd-a8c0892d0853 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.410326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.411208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.411444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.411721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.412377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.418750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.427985 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3080/6085] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:33 2022] POST /identity/v3/users => generated 281 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:33.430344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.447745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade77f5f-034f-4bd1-817e-cbb17caa19a2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.453749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3006/6086] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.454647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.454817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.456275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.456850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.457729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] PATH_INFO: `/v3/projects/1d8f0f1fdcd4411485b9986912ea4467/users/9667ff4158a44fd6aa6282e04ebd4d30/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.472736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035f3f2-03c5-41c9-946d-4fabe62fa8bc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.483993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8f0f1fdcd4411485b9986912ea4467, user_id=9667ff4158a44fd6aa6282e04ebd4d30, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.489547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.497206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.497991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.498168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.498370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.499055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.500686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665bb09-e81f-4ae6-85b9-4d2cc2fe8fe0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.510277 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3081/6087] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:33 2022] PUT /identity/v3/projects/1d8f0f1fdcd4411485b9986912ea4467/users/9667ff4158a44fd6aa6282e04ebd4d30/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:33.511940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.523300 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3007/6088] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.527950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.528234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.528510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.588177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] MFA Rules not processed for user `9667ff4158a44fd6aa6282e04ebd4d30`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:33.589588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:33.672566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde4355-dad5-4935-abda-662d2328eced admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.691376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.692270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.692554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.693744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] PATH_INFO: `/v3/projects/b4e5ba3425cf45949698d253ad1d9638/users/23910b408a864253ab77038566c6b57b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.729413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e5ba3425cf45949698d253ad1d9638, user_id=23910b408a864253ab77038566c6b57b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.731380 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3008/6089] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:33.732962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.737242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.742155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:33.755284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba7fb9-7f84-4123-a559-7c4830744952 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.761236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.763265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.763975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.764742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.765921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.768186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d591f-a4e3-4408-b662-8053f20a1ecb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:33.777494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3082/6090] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:33 2022] PUT /identity/v3/projects/b4e5ba3425cf45949698d253ad1d9638/users/23910b408a864253ab77038566c6b57b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:33.778526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:33.806929 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3009/6091] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:33.809821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.811818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.811960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.875589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] MFA Rules not processed for user `23910b408a864253ab77038566c6b57b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:33.876819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:33.924850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e12d8-c667-4034-9eea-79c6749c4c09 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:33.941166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:33.941898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:33.942752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:33.943101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:33.946279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:33.966724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.014109 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3083/6092] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:33 2022] POST /identity/v3/projects => generated 363 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:34.015363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.020484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3010/6093] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:34.021222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.038932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:34.046695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b64bb-a415-4c4d-bba4-776dc45ae07c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.115214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefa8e0-abad-4016-876e-f444d0dbcb8a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.123607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.124562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.125252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.125533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] PATH_INFO: `/v3/projects/c2762f42d4a847c9b1302ea5287469fa/users/e306103657aa476487211fbb92899ada/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.128289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.129817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.130104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.130379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.131573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.134150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.153399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2762f42d4a847c9b1302ea5287469fa, user_id=e306103657aa476487211fbb92899ada, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.157020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.177214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3084/6094] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:34 2022] PUT /identity/v3/projects/c2762f42d4a847c9b1302ea5287469fa/users/e306103657aa476487211fbb92899ada/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:34.178181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.184563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3011/6095] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:34 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:34.185862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.216450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854703c-fd99-497d-bb49-b515457c1e08 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.336739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.337726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.338855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.339398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.341268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.343793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8166f6-ea53-4ea0-aa49-3efd8a5912ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.368501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3012/6096] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:34.368501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.368501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.368884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758cbd3-5038-4d5d-82d0-31c975e16b5d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.369005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.376489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.378147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.378574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.378959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.380014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.384669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.387503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:34.407594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] MFA Rules not processed for user `e306103657aa476487211fbb92899ada`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:34.409941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:34.438608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3085/6097] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:34 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:34.440177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.523051 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3013/6098] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:34.528091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.546780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9ee9-c7c0-45f2-877c-5a2193116754 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.571954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181391a7-1f4b-4fcb-9445-f7c1d8acf2cc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.578266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.579121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.581267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.581576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] PATH_INFO: `/v3/projects/54f76b2a2eb84a2a862cce1492b23aba/users/4b13ecafa07f47ce957decc2e23d4cd5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.606531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f76b2a2eb84a2a862cce1492b23aba, user_id=4b13ecafa07f47ce957decc2e23d4cd5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.609571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.665349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.667383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.667583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.667840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.673766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.679616 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3086/6099] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:34 2022] PUT /identity/v3/projects/54f76b2a2eb84a2a862cce1492b23aba/users/4b13ecafa07f47ce957decc2e23d4cd5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:34.680576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.682041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.698863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:34.742161 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3014/6100] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:34 2022] POST /identity/v3/users => generated 298 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:34.743861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.762375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530634b1-b356-4c49-8e26-2c6f6767c4ae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.834750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51335d8-9155-415f-8852-d967d6be6448 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.842930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.844332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.845149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.846259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] PATH_INFO: `/v3/users/e306103657aa476487211fbb92899ada` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.848079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e306103657aa476487211fbb92899ada)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.850439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.878891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.879658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.880047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.880432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.881203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.883040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.894803 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3015/6101] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:34.895843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.907659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c4755-26f2-4fcb-b1a7-a5cb8ce12a17 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.913901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.914649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.915089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.915433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.916221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.918583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.933735 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3016/6102] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:34.934764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:34.934923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e306103657aa476487211fbb92899ada'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:34.935742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e306103657aa476487211fbb92899ada'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:34.937383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:34.946900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e306103657aa476487211fbb92899ada'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:34.953736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c21edb-7ffa-4236-b40b-528597ffb45f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:34.960889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:34.961880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:34.962427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:34.962886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] PATH_INFO: `/v3/users/e80162fcacaf45668a342480d116fcdc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:34.964327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80162fcacaf45668a342480d116fcdc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:34.966912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:34.973994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:34.978936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e306103657aa476487211fbb92899ada'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:34.979855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e306103657aa476487211fbb92899ada'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.008409 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3087/6103] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:34 2022] DELETE /identity/v3/users/e306103657aa476487211fbb92899ada => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:35.008409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.008409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.059747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80162fcacaf45668a342480d116fcdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.059747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80162fcacaf45668a342480d116fcdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.082880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80162fcacaf45668a342480d116fcdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.100720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80162fcacaf45668a342480d116fcdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.102627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80162fcacaf45668a342480d116fcdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:35.139317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3017/6104] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:34 2022] DELETE /identity/v3/users/e80162fcacaf45668a342480d116fcdc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:35.140149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.143496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352360d-e79a-4b0b-a69d-d3261c557fc0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:35.145425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.214597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:35.268418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258102ca-5bd3-41f8-b75f-ea957a86571c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:35.272030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:35.273563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:35.274237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.274793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.275073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] PATH_INFO: `/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c/users/fec35ffb2d7442e79fe091a880c1ee4f/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.275232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.276868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.277295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] PATH_INFO: `/v3/projects/54f76b2a2eb84a2a862cce1492b23aba/users/4b13ecafa07f47ce957decc2e23d4cd5/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.279976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f76b2a2eb84a2a862cce1492b23aba, user_id=4b13ecafa07f47ce957decc2e23d4cd5, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:35.285093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441dfd6-91e1-45b2-a90c-097ca2b3219e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:35.310433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3088/6105] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:35 2022] PUT /identity/v3/projects/54f76b2a2eb84a2a862cce1492b23aba/users/4b13ecafa07f47ce957decc2e23d4cd5/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:35.312172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22261ac8-2cc4-4cef-bd90-ac548c2e63fa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.312574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22261ac8-2cc4-4cef-bd90-ac548c2e63fa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.312954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22261ac8-2cc4-4cef-bd90-ac548c2e63fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.317237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3899fb63e2aa4c3f9d506be2cba64b6c, user_id=fec35ffb2d7442e79fe091a880c1ee4f, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:35.320038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7403b-2243-4f7f-b46d-56d37880c617 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:35.338372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-22261ac8-2cc4-4cef-bd90-ac548c2e63fa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:35.341967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-22261ac8-2cc4-4cef-bd90-ac548c2e63fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.355803 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3018/6106] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:35 2022] PUT /identity/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c/users/fec35ffb2d7442e79fe091a880c1ee4f/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:35.358100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.358391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.358688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.402018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:35.403800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.478074 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3089/6107] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:35.480674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.481277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.481731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.554931 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3019/6108] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:35.557602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.562225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] MFA Rules not processed for user `4b13ecafa07f47ce957decc2e23d4cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:35.567167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.589210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b38f54-b992-496d-b31a-509233294d1c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:35.604185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:35.606074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.606461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.610752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.612317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:35.615017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:35.623700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:35.664463 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3020/6109] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:35.667344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.673230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.787337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3090/6110] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:35.788903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.790578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:35.848634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2272b18-8674-4cc5-9385-8eaac43d7236 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:35.866406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:35.868071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:35.869526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:35.870230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:35.894324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:35.896497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:35.932717 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3021/6111] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:35.939044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:35.998775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6786c-6c05-47fc-b6d9-ba269060e852 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:36.003716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0536-6faa-4094-b830-afe60b998d77 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:36.011215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:36.012918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.014361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:36.014524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.014524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] PATH_INFO: `/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c/users/fec35ffb2d7442e79fe091a880c1ee4f/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.016017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.016491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.017206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.028033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3899fb63e2aa4c3f9d506be2cba64b6c, user_id=fec35ffb2d7442e79fe091a880c1ee4f, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:36.037959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529f3a7-3b83-4e36-a62e-025923206441 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:36.045962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:36.058217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:36.060481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:36.076380 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3091/6113] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:36.077558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:36.082254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3022/6113] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:35 2022] PUT /identity/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c/users/fec35ffb2d7442e79fe091a880c1ee4f/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:36.088912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.088912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.089264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.119991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127e4f6-df51-49a9-afad-0bf17af64a0c service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:36.138216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] MFA Rules not processed for user `fec35ffb2d7442e79fe091a880c1ee4f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:36.140630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:36.258928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:36.258928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.259291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3023/6114] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:36.262427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.264167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.271187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:36.277116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:36.427188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef17b96-7ab9-4bbe-b45e-4bde62c720aa None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:36.432424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:36.433078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:36.433226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.433539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.433817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.435639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:36.451977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:36.453641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:36.460978 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3092/6115] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:36.490919 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3024/6116] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:36.637154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:36.655998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c2c3e-b2ac-4d7d-b36c-7e33489d5654 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:36.664150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:36.664787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:36.665067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:36.665342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:36.667734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:36.772698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:36.775203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:36.790984 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3093/6117] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:37.239282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:37.241052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:37.290305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:37.359918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44801fce-694d-4730-85ae-c1c3e7da80e5 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:37.365844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:37.366690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:37.367166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:37.367531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] PATH_INFO: `/v3/projects/829a61ff30bc493397b34887fdb0e65e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:37.368754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=829a61ff30bc493397b34887fdb0e65e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:37.370946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:37.449289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829a61ff30bc493397b34887fdb0e65e', 'request_id': 'req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.457068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829a61ff30bc493397b34887fdb0e65e', 'request_id': 'req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.467132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829a61ff30bc493397b34887fdb0e65e', 'request_id': 'req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.481467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] The token cache is being invalidate because project 829a61ff30bc493397b34887fdb0e65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:37.481763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.482971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.484708 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3025/6118] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:37 2022] DELETE /identity/v3/projects/829a61ff30bc493397b34887fdb0e65e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:37.488737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:37.501035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b590181-1fb8-4860-a81a-fed7159ff853 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:37.553069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:37.577517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:37.578593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:37.578593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:37.578961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] PATH_INFO: `/v3/users/b3fccd0dc21c46f5a1c7d91d9514e71b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:37.580822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fccd0dc21c46f5a1c7d91d9514e71b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:37.583068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:37.657265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fccd0dc21c46f5a1c7d91d9514e71b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.658300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fccd0dc21c46f5a1c7d91d9514e71b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.670755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fccd0dc21c46f5a1c7d91d9514e71b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.687247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fccd0dc21c46f5a1c7d91d9514e71b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.688038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fccd0dc21c46f5a1c7d91d9514e71b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:37.702800 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3094/6119] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:37 2022] DELETE /identity/v3/users/b3fccd0dc21c46f5a1c7d91d9514e71b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:37.782872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:37.785859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:37.803722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cf7d7-0527-4ae6-9a9f-b472a61e52fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:37.907483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:37.909392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:37.909868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:37.910321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] PATH_INFO: `/v3/projects/c2762f42d4a847c9b1302ea5287469fa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:37.911640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2762f42d4a847c9b1302ea5287469fa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:37.914693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:37.922387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:37.924541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:38.023532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:38.028513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2762f42d4a847c9b1302ea5287469fa', 'request_id': 'req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.040047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2762f42d4a847c9b1302ea5287469fa', 'request_id': 'req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.053756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2762f42d4a847c9b1302ea5287469fa', 'request_id': 'req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.076368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd9cd-b6ba-4ba3-abcf-7cdb4eaaf5d6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.088442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] The token cache is being invalidate because project c2762f42d4a847c9b1302ea5287469fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:38.088871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.089701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-851b7c8b-b1e4-4579-a0ab-bec7919e7a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.100650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3026/6120] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:37 2022] DELETE /identity/v3/projects/c2762f42d4a847c9b1302ea5287469fa => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:38.102467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdecbd-1328-4c61-9865-af4f8cd57db3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.102845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdecbd-1328-4c61-9865-af4f8cd57db3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.103183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdecbd-1328-4c61-9865-af4f8cd57db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.147016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d9fdecbd-1328-4c61-9865-af4f8cd57db3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:38.149192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9fdecbd-1328-4c61-9865-af4f8cd57db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:38.214146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b469649953a24842839d689b1f3885be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57580819362741b182ab7616cb8793ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.215865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.216498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.217031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.218301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:38.219831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.232630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.250291 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3095/6121] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:38.256216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.258816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:38.277569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:38.285387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8069c-58f1-447d-9fc7-07aa25bb5feb tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.294099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b469649953a24842839d689b1f3885be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57580819362741b182ab7616cb8793ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.295389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.302461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.304419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.305116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.309464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.313234 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3027/6122] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:38.334368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.335004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.335586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.394772 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3096/6123] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:38 2022] POST /identity/v3/projects => generated 347 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:38.399000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.400519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:38.404922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:38.547480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba794cb2-910e-4785-8a7c-445d503d9f53 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.561479 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3028/6124] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:38.567749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.567983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b469649953a24842839d689b1f3885be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57580819362741b182ab7616cb8793ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.568696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.569102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.569596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.570453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:38.577011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.579195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.589451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d0f81-c1d1-4b70-94ec-630e2cc4ffb2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.600241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.602190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.602490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.604161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.605881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:38.607426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.612553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.614588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3097/6125] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:12:38.650197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3029/6126] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:38.653534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.681781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648800-057c-4527-b1f8-d4a85ad92aa5 tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.696537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.697088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.698207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.698617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.698985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.700069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.700522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:38.708855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.723588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6084dc3-1b33-43ec-92de-f00e5694d3d4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:38.731063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:38.732590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:38.733210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:38.733745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] PATH_INFO: `/v3/users/23910b408a864253ab77038566c6b57b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:38.737078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23910b408a864253ab77038566c6b57b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:38.740433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:38.787491 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3098/6127] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:38 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:38.789498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:38.856997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23910b408a864253ab77038566c6b57b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.859309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23910b408a864253ab77038566c6b57b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.896934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23910b408a864253ab77038566c6b57b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.901178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:38.912514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23910b408a864253ab77038566c6b57b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:38.920632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b8d820ae-8ad4-47df-aeb2-222ea7a508f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23910b408a864253ab77038566c6b57b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.003306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3030/6128] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:38 2022] DELETE /identity/v3/users/23910b408a864253ab77038566c6b57b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:39.034651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.043450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.046156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.087557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525903a4-c262-486d-ada6-8cd29623085e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.175452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:39.187722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.293094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.293724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.294704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.294986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.295628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.297714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.308114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:39.339032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3099/6129] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:38 2022] POST /identity/v3/users => generated 284 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:39.340190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.346163 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3031/6130] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:39.347847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.348259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.349645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.365657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e590-1f11-4527-a129-1e9603a2fac4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.366425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1df6e-32c5-4515-a622-a2a1ff6a31e9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.372521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.373388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.375814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.377330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.377698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.378012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.378351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] PATH_INFO: `/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.378719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.380015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.380498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e693ae1e6194ea9a07e54f3beaedbbd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.382156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.383673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.397324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3032/6131] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:39.403313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.420895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81aac43-f387-45c0-9bb2-0a2919918795 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.435935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.436486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.436769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.437046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.437699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.444964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.472220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3033/6132] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:39.472220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.487640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.540728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e693ae1e6194ea9a07e54f3beaedbbd', 'request_id': 'req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.556378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e693ae1e6194ea9a07e54f3beaedbbd', 'request_id': 'req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.586035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e693ae1e6194ea9a07e54f3beaedbbd', 'request_id': 'req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.618920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] The token cache is being invalidate because project 3e693ae1e6194ea9a07e54f3beaedbbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:39.619281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.622359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:39.663516 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3100/6133] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:39 2022] DELETE /identity/v3/projects/3e693ae1e6194ea9a07e54f3beaedbbd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:39.664744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.667248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.675668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45529800-907c-4dbf-a345-caf58c4457d5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.769539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab6ee7-715b-4111-8f0b-d6d71e207d6c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:39.778654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.780425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.780901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.781540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] PATH_INFO: `/v3/projects/11e1c84367ae456084d07ea54f6df68a/users/a9fe52d782a54802aac19a97e4517de7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.796207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:39.798065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.800173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.800581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.801914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e1c84367ae456084d07ea54f6df68a, user_id=a9fe52d782a54802aac19a97e4517de7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.804882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d384733-74d9-4696-a702-02d281dca0c6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.805778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.825959 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3101/6134] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:39 2022] PUT /identity/v3/projects/11e1c84367ae456084d07ea54f6df68a/users/a9fe52d782a54802aac19a97e4517de7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:39.828325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.830488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.830961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.833174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:39.833826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eef32-49fd-4d77-86c0-6e2dff6beb65 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:39.864584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3034/6135] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:39.870144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710abcd9-d4eb-46ad-9f27-44bf717477aa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:39.870674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710abcd9-d4eb-46ad-9f27-44bf717477aa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:39.873257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710abcd9-d4eb-46ad-9f27-44bf717477aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:39.893692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:39.894992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.926284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:39.941354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-710abcd9-d4eb-46ad-9f27-44bf717477aa None None] MFA Rules not processed for user `a9fe52d782a54802aac19a97e4517de7`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:39.948930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-710abcd9-d4eb-46ad-9f27-44bf717477aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:39.976269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:39.991458 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3102/6136] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:39.995651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:39.999966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.058392 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3035/6137] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:40.058392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.058392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.059646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.096719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4649d-5135-495a-a63c-30b62b7d2f5b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.102224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.102889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.103170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.103441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.105231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.105862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:40.107570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.126626 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3036/6138] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:40.126626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.148845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dff99-9c13-419f-8ba1-1c1567a0ab27 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.157341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.159432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.159892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.160473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.161615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.164533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.175274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.176728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.187793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3103/6139] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:40.195695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.200677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.212941 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3037/6140] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:40 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:40.212941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.302537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079431f-8256-472b-9f28-9d969c7acef9 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.323203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65784175-bc46-4820-b87d-dc97c3551350 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.334753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.336982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.337509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.338025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.339282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:40.340081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.348046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.390438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:40.397354 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3038/6141] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:40.398714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.405994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b469649953a24842839d689b1f3885be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57580819362741b182ab7616cb8793ab', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.406677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.406975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.407242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] PATH_INFO: `/v3/projects/d36cdac46fb448ba8f9f60b8c49919ae` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.414165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791c4a5-b12b-437f-a9dc-3ac6c6bc2334 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.415728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d36cdac46fb448ba8f9f60b8c49919ae)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.418584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.430334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.431375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.431660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.431931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.432560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.434681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.464529 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3039/6142] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:40 2022] POST /identity/v3/users => generated 291 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:40.465769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.480377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9df177-cfca-4aa1-8b87-c1bfce5d5e67 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.487894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.488617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.488905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.489673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.489885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.492157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.510645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36cdac46fb448ba8f9f60b8c49919ae', 'request_id': 'req-65e30f01-c4a6-4842-a15e-d2819a438a3c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:40.518860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36cdac46fb448ba8f9f60b8c49919ae', 'request_id': 'req-65e30f01-c4a6-4842-a15e-d2819a438a3c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:40.533796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36cdac46fb448ba8f9f60b8c49919ae', 'request_id': 'req-65e30f01-c4a6-4842-a15e-d2819a438a3c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:40.535853 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3040/6143] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:40 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:40.540485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.558421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] The token cache is being invalidate because project d36cdac46fb448ba8f9f60b8c49919ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:40.558819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:40.560538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:40.562748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3104/6144] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:40 2022] DELETE /identity/v3/projects/d36cdac46fb448ba8f9f60b8c49919ae => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:40.563841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.567045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.589465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:40.660294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13f9c-94dd-410c-b5fb-ffb0ea2d3ed2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.676204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.677500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e30f01-c4a6-4842-a15e-d2819a438a3c tempest-RoutersIpV6AdminTest-1521593319 tempest-RoutersIpV6AdminTest-1521593319-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.679178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.679558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.679845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.680511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.682920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.683644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.684019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.684169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.684465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.685688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.688068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.703486 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3041/6145] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:40.704432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.707147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.723707 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3105/6146] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:40 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:40.724918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.727080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.757575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc896c-1dbb-41a2-b45c-d27d6046b89d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.757575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.757575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.757575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.758255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] PATH_INFO: `/v3/projects/61733d1dfced469c82333e9e63905bf0/users/f7885acf68f04513964ccd32657f5534/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.784071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=61733d1dfced469c82333e9e63905bf0, user_id=f7885acf68f04513964ccd32657f5534, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.788927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.792402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670dc7-41f5-4dc2-9ac8-47e473205928 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.800061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:40.800680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:40.801147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:40.801311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:40.805507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:40.812198 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3106/6147] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:40 2022] PUT /identity/v3/projects/61733d1dfced469c82333e9e63905bf0/users/f7885acf68f04513964ccd32657f5534/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:40.817336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.913025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:40.914663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:40.926885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3042/6148] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:40.927679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:40.945000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915a01-1aa5-4722-94d1-379e17bde851 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:40.968193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:40.997183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402bdc9-15d2-4ff6-a0ca-4670767ad76d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.018976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.019955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.020362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.020641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.021292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.023305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.033462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.034347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.034937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.035425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.036243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.038677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.041119 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3107/6149] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:41.042063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.044520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.054971 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3043/6150] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:41.056094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.067695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1316e-79ab-482f-b84f-5917b9ad4ca3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.072143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ca809-80ee-4ab1-a498-34808ac552a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.076342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.076925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.077176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.077600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] PATH_INFO: `/v3/projects/b4e5ba3425cf45949698d253ad1d9638` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.080726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.081123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e5ba3425cf45949698d253ad1d9638)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.082258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.082765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.083250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.083498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.084429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.086597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.100935 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3044/6151] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:41.102439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.108320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:41.118947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb500752-7ee1-4076-9a94-640fadc9be67 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.125639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.126757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.127468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.127947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] PATH_INFO: `/v3/projects/61733d1dfced469c82333e9e63905bf0/users/f7885acf68f04513964ccd32657f5534/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.130395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61733d1dfced469c82333e9e63905bf0, user_id=f7885acf68f04513964ccd32657f5534, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.134891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.152277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e5ba3425cf45949698d253ad1d9638', 'request_id': 'req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.162970 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3045/6152] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:41 2022] PUT /identity/v3/projects/61733d1dfced469c82333e9e63905bf0/users/f7885acf68f04513964ccd32657f5534/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:41.162970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.168069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e5ba3425cf45949698d253ad1d9638', 'request_id': 'req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.176797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e5ba3425cf45949698d253ad1d9638', 'request_id': 'req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.196422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] The token cache is being invalidate because project b4e5ba3425cf45949698d253ad1d9638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:41.197082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.198041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-18a65cc7-1fdb-4f9c-983d-6ee526cbdf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.203510 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3108/6153] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:41 2022] DELETE /identity/v3/projects/b4e5ba3425cf45949698d253ad1d9638 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:41.207687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208621-9ff1-4380-8c8d-155191ad4acf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.208244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208621-9ff1-4380-8c8d-155191ad4acf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.208796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208621-9ff1-4380-8c8d-155191ad4acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.246210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-09208621-9ff1-4380-8c8d-155191ad4acf None None] MFA Rules not processed for user `f7885acf68f04513964ccd32657f5534`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:41.248252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-09208621-9ff1-4380-8c8d-155191ad4acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.251270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a40813-44f5-46bd-bbfc-53f45f8e0137 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.258053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.258841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.259108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.259532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] PATH_INFO: `/v3/projects/10302b5a9dfe48ec96bd17b480c2c7b6/users/55e3a217666f4b9faa2650b7e75761e2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.282269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10302b5a9dfe48ec96bd17b480c2c7b6, user_id=55e3a217666f4b9faa2650b7e75761e2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.286605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.307384 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3046/6154] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:41 2022] PUT /identity/v3/projects/10302b5a9dfe48ec96bd17b480c2c7b6/users/55e3a217666f4b9faa2650b7e75761e2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:41.308273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.310528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.356543 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3109/6155] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:41.359192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.359861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.360567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.401788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] MFA Rules not processed for user `55e3a217666f4b9faa2650b7e75761e2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:41.403454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.403739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62991-c249-4948-ab42-71fe74413b32 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.411431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.412117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.412384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.412907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] PATH_INFO: `/v3/users/fec35ffb2d7442e79fe091a880c1ee4f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.415523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec35ffb2d7442e79fe091a880c1ee4f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.417525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.486745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:41.488343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec35ffb2d7442e79fe091a880c1ee4f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.489050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec35ffb2d7442e79fe091a880c1ee4f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.500208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec35ffb2d7442e79fe091a880c1ee4f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.510121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec35ffb2d7442e79fe091a880c1ee4f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.510980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec35ffb2d7442e79fe091a880c1ee4f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.512987 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3110/6156] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:41.513711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.515110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.531323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:41.549119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3047/6157] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:41 2022] DELETE /identity/v3/users/fec35ffb2d7442e79fe091a880c1ee4f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:41.549119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.561341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.599006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a44e1-247f-456c-8885-d019eb618ac7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.670445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd048f-1143-41fb-a236-1e959fe3e75a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.677025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.677671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.678102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.678372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] PATH_INFO: `/v3/users/9667ff4158a44fd6aa6282e04ebd4d30` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.679552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9667ff4158a44fd6aa6282e04ebd4d30)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.681974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.700082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.701310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.701874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.702304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.704727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.788830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9667ff4158a44fd6aa6282e04ebd4d30'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.789675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9667ff4158a44fd6aa6282e04ebd4d30'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.809599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9667ff4158a44fd6aa6282e04ebd4d30'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.821892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9667ff4158a44fd6aa6282e04ebd4d30'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.822952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9667ff4158a44fd6aa6282e04ebd4d30'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:41.850546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:41.856729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75425bd2-6cbd-4d8b-9050-3918171bd370 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:41.876554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3048/6158] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:41 2022] DELETE /identity/v3/users/9667ff4158a44fd6aa6282e04ebd4d30 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:41.877511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:41.884971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.891360 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3111/6159] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:41.893917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.894277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.894547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.921587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:41.923180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:41.979726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:41.984106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd345e15-2e33-417f-9bfe-4f46b0380090 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:41.991237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:41.992197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:41.992724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:41.993315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:41.995575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:42.012230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3112/6160] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:42.032947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:42.075867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:42.079462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:42.110282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.115087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.141697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3049/6161] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:42.143506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:42.146633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:42.180158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71f33-7b4d-4f1d-a900-448ee0b5595c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:42.184597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eb584-5735-44a3-ad1a-42f34a1a8fec service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:42.184742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:42.185597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.185977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.186459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.197372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:42.198232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.198540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.198824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] PATH_INFO: `/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.200868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.201012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3899fb63e2aa4c3f9d506be2cba64b6c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.202867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.203976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec71b5b-3963-4a9d-8a56-1f6412aea9a3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.218823 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3113/6162] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:42.309920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3899fb63e2aa4c3f9d506be2cba64b6c', 'request_id': 'req-393df9e7-0762-441f-b4b3-f83c4115c60b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:42.322825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3899fb63e2aa4c3f9d506be2cba64b6c', 'request_id': 'req-393df9e7-0762-441f-b4b3-f83c4115c60b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:42.337473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3899fb63e2aa4c3f9d506be2cba64b6c', 'request_id': 'req-393df9e7-0762-441f-b4b3-f83c4115c60b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:42.364308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] The token cache is being invalidate because project 3899fb63e2aa4c3f9d506be2cba64b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:42.364636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:42.365267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-393df9e7-0762-441f-b4b3-f83c4115c60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:42.372113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3050/6163] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:42 2022] DELETE /identity/v3/projects/3899fb63e2aa4c3f9d506be2cba64b6c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:42.374320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.374320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.374320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.397331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:42.398918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:42.514354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.514942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.515147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.524515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3114/6164] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:12:42.546133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:42.549418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:42.568959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:42.576084 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3051/6165] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:42.581813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6162bdc-d915-4b23-bb57-6e459b071b7c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:42.594536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:42.598368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.598932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.599219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.602436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:42.602922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.609391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.645298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:42.654060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3115/6166] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:42.656439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544688-4fb5-4e31-8e3c-4adf89959f20 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:42.670655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:42.671281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.671561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.671832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.672463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.674659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.726735 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3052/6167] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:42 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:42.734935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:42.748644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852edc7d-89da-4778-91f8-734a4dd4bba7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:42.905180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:42.905180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:42.905180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:42.905180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:42.906209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:42.906209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:42.966542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3116/6168] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:42 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:42.968247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.002641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92839a66-c03a-4d0f-8534-d1a3e0f92fde admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.015955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.017035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.017717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.018243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.019356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.025322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.049309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3053/6169] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:43.050433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.065332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3bdb7-3fd9-4f40-8d03-86c2cd5242fd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.072488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.075629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.075919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.076199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.076859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.079373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340d73-2147-452f-ad7a-43150b545674 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.095936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3117/6170] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:12:43.100843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.130474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a8992-c52f-4f47-acad-10ae1182ee6c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.136935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.137611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.137893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.138171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] PATH_INFO: `/v3/projects/ac17de168d364cfca453c2eec2529ac2/users/4cd6575e42b0433a9d5be95b34bc4a8f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.158174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac17de168d364cfca453c2eec2529ac2, user_id=4cd6575e42b0433a9d5be95b34bc4a8f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.160936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.178612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3054/6171] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:43 2022] PUT /identity/v3/projects/ac17de168d364cfca453c2eec2529ac2/users/4cd6575e42b0433a9d5be95b34bc4a8f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:43.189428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.190691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.191374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.202867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:43.232002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] MFA Rules not processed for user `4cd6575e42b0433a9d5be95b34bc4a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:43.233245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:43.308626 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3118/6172] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:43.313460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.369174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:43.405077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19616-857e-4a5b-8d88-af7dbb62e528 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.411239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.412162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.412511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.412859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.413835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.415994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.454293 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3055/6173] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:43 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:43.459144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.480563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8488e-0d86-48ed-bfbc-9515eab509c9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.567638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.570027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.570539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.571477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.572798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.581035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.583756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:43.593703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.619645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a95b81-9d70-4bbe-a7d4-33734b1e2a8e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.632184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.632836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.633117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.633396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] PATH_INFO: `/v3/projects/1d8f0f1fdcd4411485b9986912ea4467` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.636139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8f0f1fdcd4411485b9986912ea4467)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.638382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.654158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3119/6174] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:43 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:12:43.660550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.673889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371f987-1669-4a38-8af3-7f610e9346d4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.683340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.684127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.685328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.685801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.686454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.688242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.704014 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3120/6175] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:12:43.711421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.726487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2425caf-78d8-40c1-930d-ff7d90d3faca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.726738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8f0f1fdcd4411485b9986912ea4467', 'request_id': 'req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.736800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8f0f1fdcd4411485b9986912ea4467', 'request_id': 'req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.748050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8f0f1fdcd4411485b9986912ea4467', 'request_id': 'req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.764242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] The token cache is being invalidate because project 1d8f0f1fdcd4411485b9986912ea4467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:43.764506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.765197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.772269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3056/6176] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:43 2022] DELETE /identity/v3/projects/1d8f0f1fdcd4411485b9986912ea4467 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:43.773333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.777397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:43.845441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.846321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.846631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.846968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.848546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.851867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.869039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3121/6177] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:43.873957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:43.874217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e0600-7c31-4f9c-9d1d-190ef6fbec0b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.876296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:43.880091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.880788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.881125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.881406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] PATH_INFO: `/v3/users/4b13ecafa07f47ce957decc2e23d4cd5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.882726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b13ecafa07f47ce957decc2e23d4cd5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.884674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.899063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954052-e9c0-4d5f-852a-eb6bd8fb7cea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:43.906811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:43.907920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.908802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.909765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] PATH_INFO: `/v3/projects/771d8a6abcf1416e970d19968c258f02/users/1d2bb0bc7aa7443e85dcc7570f224141/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:43.920424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:43.949860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] RBAC: Authorizing `identity:create_grant(project_id=771d8a6abcf1416e970d19968c258f02, user_id=1d2bb0bc7aa7443e85dcc7570f224141, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:43.957615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a658dcc-fae8-459c-8046-f5561c8f713c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:43.980976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3122/6178] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:43 2022] PUT /identity/v3/projects/771d8a6abcf1416e970d19968c258f02/users/1d2bb0bc7aa7443e85dcc7570f224141/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:43.981417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b13ecafa07f47ce957decc2e23d4cd5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.982129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b13ecafa07f47ce957decc2e23d4cd5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:43.994816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:43.995201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:43.995540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.012850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b13ecafa07f47ce957decc2e23d4cd5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.016211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:44.054925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b13ecafa07f47ce957decc2e23d4cd5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.065196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b13ecafa07f47ce957decc2e23d4cd5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.103909 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3057/6179] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:43 2022] DELETE /identity/v3/users/4b13ecafa07f47ce957decc2e23d4cd5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:44.124483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] MFA Rules not processed for user `1d2bb0bc7aa7443e85dcc7570f224141`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:44.128079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:44.291118 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3123/6180] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:44.295948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:44.297870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:44.374785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:44.376254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:44.462709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad265ce5-fb1e-40a1-848a-154cdc33a63a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:44.475072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:44.476475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.476764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.477046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.477717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:44.481649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:44.516633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca5439-839d-4786-8632-6ebbe54fc4d8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:44.525836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3058/6181] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:44 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:44.527397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:44.531912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:44.533144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.533514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.534138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] PATH_INFO: `/v3/users/55e3a217666f4b9faa2650b7e75761e2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.537157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e3a217666f4b9faa2650b7e75761e2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:44.539249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:44.665974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e3a217666f4b9faa2650b7e75761e2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.666736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e3a217666f4b9faa2650b7e75761e2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.685547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e3a217666f4b9faa2650b7e75761e2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.692473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e1f1-b6d4-4cf5-aa95-3bfd6c2d3540 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:44.704384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:44.706279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.706626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.706933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.707647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:44.708943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e3a217666f4b9faa2650b7e75761e2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.712751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bdf731f9-ae93-45ae-abef-0325f34fa8e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e3a217666f4b9faa2650b7e75761e2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:44.716498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:44.722739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:44.737798 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3124/6182] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:44 2022] DELETE /identity/v3/users/55e3a217666f4b9faa2650b7e75761e2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:44.745020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.745878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.746166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.762558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3059/6183] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:44 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:44.766628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:44.775438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:44.784502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:44.786848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:44.791533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7db4f-bf71-42b6-87e9-f0af36b74d7a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:44.853631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:44.869380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:44.870046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.870828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.872073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.872860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:44.874960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:44.894681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3060/6184] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:44.895655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:44.910758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a7c2c-1a5c-492a-a6ad-44f8ca70d569 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:44.916984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:44.917641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:44.918416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:44.923773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] PATH_INFO: `/v3/projects/57256c04221643029b91ab7a3aa82819/users/447fe4899e8b425b8db7c70d8f920a91/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:44.930360 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3125/6185] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:44.955300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57256c04221643029b91ab7a3aa82819, user_id=447fe4899e8b425b8db7c70d8f920a91, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:44.959919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:44.989988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3061/6186] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:44 2022] PUT /identity/v3/projects/57256c04221643029b91ab7a3aa82819/users/447fe4899e8b425b8db7c70d8f920a91/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:44.990966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.004124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f28b30-401d-4550-a025-af4f5a3dfefa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.004448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.008227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.129872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125418ea-254e-4a5f-a51c-a42211f75d82 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.134077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.134883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.135208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.135423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.144732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.145495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.145779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.146158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.146921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.148972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.149689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.152361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.161559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3126/6187] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:45.163135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.165529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.175815 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3062/6188] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:45.176571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.182694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce5c27-352d-4ccd-b2b8-0ea65e72ff96 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.187627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4cf52-aad7-49c7-82b6-69370b46b073 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.187803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.188523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.189481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.189804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] PATH_INFO: `/v3/users/a9fe52d782a54802aac19a97e4517de7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.192061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9fe52d782a54802aac19a97e4517de7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.194063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.195133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.195789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.196188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.196357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] PATH_INFO: `/v3/projects/57256c04221643029b91ab7a3aa82819/users/447fe4899e8b425b8db7c70d8f920a91/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.198380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=57256c04221643029b91ab7a3aa82819, user_id=447fe4899e8b425b8db7c70d8f920a91, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.201231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d8d92-c849-4af3-b36e-1414885319ee admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.215289 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3063/6189] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:45 2022] PUT /identity/v3/projects/57256c04221643029b91ab7a3aa82819/users/447fe4899e8b425b8db7c70d8f920a91/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:45.225826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.226574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.227167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.280595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9fe52d782a54802aac19a97e4517de7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.281308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9fe52d782a54802aac19a97e4517de7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.284211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] MFA Rules not processed for user `447fe4899e8b425b8db7c70d8f920a91`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:45.285708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.291821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9fe52d782a54802aac19a97e4517de7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.315430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9fe52d782a54802aac19a97e4517de7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.317269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-76d0c96f-ff21-4ff0-8a5f-a27ad42376bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9fe52d782a54802aac19a97e4517de7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.330122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:45.354983 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3127/6190] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:45 2022] DELETE /identity/v3/users/a9fe52d782a54802aac19a97e4517de7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:45.359550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.360783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.362026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.425130 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3064/6191] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:45.426967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.436860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:45.438602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.441114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.499900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:45.546997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d4f9c-842c-4ff3-b093-5a8440eace7b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.561308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3128/6192] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:45.562930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.565262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.636777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.637586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.637987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.638453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] PATH_INFO: `/v3/users/325f5dc462514e8e85d5adf9d433cb40` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.641403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=325f5dc462514e8e85d5adf9d433cb40)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.645592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.653017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36710455-40e1-46de-a99d-7fb0be868d53 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.661331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.664743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.665165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.666084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.741703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325f5dc462514e8e85d5adf9d433cb40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.742800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325f5dc462514e8e85d5adf9d433cb40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.755626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325f5dc462514e8e85d5adf9d433cb40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.767006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325f5dc462514e8e85d5adf9d433cb40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.767859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325f5dc462514e8e85d5adf9d433cb40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:45.780742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.783275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.797304 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3129/6193] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:45.798914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.799970 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3065/6194] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:45 2022] DELETE /identity/v3/users/325f5dc462514e8e85d5adf9d433cb40 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:45.800819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.801645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:45.803328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:45.906592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b27346-3c5f-47d1-9c77-243bda0d723f service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.908715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959eb0e-f2bc-4f29-9d52-ca81efc1a19c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:45.916824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:45.926628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.927397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.928445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.929006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] PATH_INFO: `/v3/projects/54f76b2a2eb84a2a862cce1492b23aba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.931136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f76b2a2eb84a2a862cce1492b23aba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:45.933770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:45.935952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:45.989216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:45.990454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:45.990991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:45.991752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:45.996147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.013737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.015238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce329-231e-48b4-b887-bce20f72920e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.023110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f76b2a2eb84a2a862cce1492b23aba', 'request_id': 'req-482b2fcd-7117-47ff-b578-723df631bf87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.028831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3130/6195] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:46.030184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.030472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.030630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f76b2a2eb84a2a862cce1492b23aba', 'request_id': 'req-482b2fcd-7117-47ff-b578-723df631bf87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.030784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.041933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f76b2a2eb84a2a862cce1492b23aba', 'request_id': 'req-482b2fcd-7117-47ff-b578-723df631bf87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.052421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:46.053714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.061156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] The token cache is being invalidate because project 54f76b2a2eb84a2a862cce1492b23aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:46.061779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.063074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.066475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3066/6196] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:45 2022] DELETE /identity/v3/projects/54f76b2a2eb84a2a862cce1492b23aba => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:46.068119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.070475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.164746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b2fcd-7117-47ff-b578-723df631bf87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.172695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.174588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.174920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.176450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.181291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.182699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3131/6197] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:46.183977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.187489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.219603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8cd1-a172-4f56-a670-2d6b1deb9875 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.230155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.231473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.231954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.232780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] PATH_INFO: `/v3/projects/10302b5a9dfe48ec96bd17b480c2c7b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.235595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10302b5a9dfe48ec96bd17b480c2c7b6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.239244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.338839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10302b5a9dfe48ec96bd17b480c2c7b6', 'request_id': 'req-eb96317a-3f86-41ab-a81f-39b769859f22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.353933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.359798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c44bc-b2aa-4abd-afde-b5e7578b96fe service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.363109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10302b5a9dfe48ec96bd17b480c2c7b6', 'request_id': 'req-eb96317a-3f86-41ab-a81f-39b769859f22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.375184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10302b5a9dfe48ec96bd17b480c2c7b6', 'request_id': 'req-eb96317a-3f86-41ab-a81f-39b769859f22'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.385476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3067/6198] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:46.387314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22273ec6-2486-4ff2-a235-e6ddcd065c2e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.387766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22273ec6-2486-4ff2-a235-e6ddcd065c2e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.388246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22273ec6-2486-4ff2-a235-e6ddcd065c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.442227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] The token cache is being invalidate because project 10302b5a9dfe48ec96bd17b480c2c7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:46.443622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-22273ec6-2486-4ff2-a235-e6ddcd065c2e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:46.447727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.448581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-22273ec6-2486-4ff2-a235-e6ddcd065c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.450326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:46.452780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3132/6199] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:46 2022] DELETE /identity/v3/projects/10302b5a9dfe48ec96bd17b480c2c7b6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:46.457540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.463413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.548816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:46.589779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96317a-3f86-41ab-a81f-39b769859f22 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.591208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:46.603928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.607367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.608349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.608814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.614347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.665142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3068/6200] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:46.665142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb996740-41f3-4a56-8d33-aecf1cb3f56b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.665142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb996740-41f3-4a56-8d33-aecf1cb3f56b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.665142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb996740-41f3-4a56-8d33-aecf1cb3f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.712006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bb996740-41f3-4a56-8d33-aecf1cb3f56b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:46.713248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bb996740-41f3-4a56-8d33-aecf1cb3f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.742115 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3069/6201] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:46.745319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c38b8-0cd4-427a-93a0-1c46cb80424b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.745610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c38b8-0cd4-427a-93a0-1c46cb80424b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.745878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c38b8-0cd4-427a-93a0-1c46cb80424b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.782905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ac8c38b8-0cd4-427a-93a0-1c46cb80424b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:46.784330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ac8c38b8-0cd4-427a-93a0-1c46cb80424b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.801008 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3070/6202] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:46.803369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371fc25-14ab-4688-be30-08438791574d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.803826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371fc25-14ab-4688-be30-08438791574d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.804382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371fc25-14ab-4688-be30-08438791574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.834760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d371fc25-14ab-4688-be30-08438791574d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:46.838655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d371fc25-14ab-4688-be30-08438791574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.843674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.848855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.863116 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3071/6203] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:46.865668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371fc25-14ab-4688-be30-08438791574d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.870388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d371fc25-14ab-4688-be30-08438791574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:46.872604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3133/6204] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:46.873713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.909981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371fc25-14ab-4688-be30-08438791574d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.911468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a11be-9249-44b7-a8c0-0b2d0c4bcf3d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.923022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.924025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.924988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.925803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.926077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.926467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.926792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] PATH_INFO: `/v3/projects/11e1c84367ae456084d07ea54f6df68a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.927499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.929154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e1c84367ae456084d07ea54f6df68a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.929663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:46.930077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.933692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.934215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:46.963191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3134/6205] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:46.964030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:46.977660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9c71-1843-409a-8538-4367b2279a67 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:46.984454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:46.985126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:46.986522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:46.986709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:46.987427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:46.987921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:46.989862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.006239 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3135/6206] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:47.006239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.018132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e1c84367ae456084d07ea54f6df68a', 'request_id': 'req-303a2ca2-69cb-43dd-aabc-8abb6b316494'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.024317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35899239-6563-4d96-a18b-e368b93e0358 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.025357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e1c84367ae456084d07ea54f6df68a', 'request_id': 'req-303a2ca2-69cb-43dd-aabc-8abb6b316494'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.036811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e1c84367ae456084d07ea54f6df68a', 'request_id': 'req-303a2ca2-69cb-43dd-aabc-8abb6b316494'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.054706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] The token cache is being invalidate because project 11e1c84367ae456084d07ea54f6df68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:47.055091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.055931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.058027 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3072/6207] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:46 2022] DELETE /identity/v3/projects/11e1c84367ae456084d07ea54f6df68a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:47.058904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.060350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.151924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.152544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.152822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.153092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.154020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.156633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.195907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a2ca2-69cb-43dd-aabc-8abb6b316494 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.202581 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3136/6208] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/projects => generated 397 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:47.202581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.210565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.218593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.219239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.219522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.219801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.221013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.223118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.280962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:47.281537 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3073/6209] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/projects => generated 389 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:47.285957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.288873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.321573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eef0b-7901-483c-9a08-a2c88dd8b60e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.341575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:47.390873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a34c-9fc0-4daa-a502-ba85bd1b7a0c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.396981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.397588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.397868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.398317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.399101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.404202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.415277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.416957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.417313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.417611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] PATH_INFO: `/v3/users/f7885acf68f04513964ccd32657f5534` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.419653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7885acf68f04513964ccd32657f5534)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.421767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.437393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3074/6210] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:47.439901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.456171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dae0f-ac17-4e8c-9154-a3f99e73ec84 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.462325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.463371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.466619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.466619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.466619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.466619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.496869 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3075/6211] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/users => generated 295 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:47.497808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.499908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.503683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7885acf68f04513964ccd32657f5534'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.504538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7885acf68f04513964ccd32657f5534'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.513404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7885acf68f04513964ccd32657f5534'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.514918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba733e-cc09-41b4-a88e-db384c62b4fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.524768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.525496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.525851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.526212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] PATH_INFO: `/v3/projects/ab991a36186f40b2bee70625b3f7424c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.527475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab991a36186f40b2bee70625b3f7424c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.527811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7885acf68f04513964ccd32657f5534'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.528599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7885acf68f04513964ccd32657f5534'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.533563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.543266 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3137/6212] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:47 2022] DELETE /identity/v3/users/f7885acf68f04513964ccd32657f5534 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:47.546562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.559622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.591562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab991a36186f40b2bee70625b3f7424c', 'request_id': 'req-260ce056-d06e-4f88-84ee-de7bfaed7b42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.599341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab991a36186f40b2bee70625b3f7424c', 'request_id': 'req-260ce056-d06e-4f88-84ee-de7bfaed7b42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.609917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab991a36186f40b2bee70625b3f7424c', 'request_id': 'req-260ce056-d06e-4f88-84ee-de7bfaed7b42'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.628604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] The token cache is being invalidate because project ab991a36186f40b2bee70625b3f7424c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:47.628900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.630271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:47.632432 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3076/6213] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:47 2022] DELETE /identity/v3/projects/ab991a36186f40b2bee70625b3f7424c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:47.636797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.639049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.679993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed9ad-d615-49df-aefb-e8a13919c930 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.698769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:47.753526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ce056-d06e-4f88-84ee-de7bfaed7b42 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.759817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.760572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.760905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.761244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.761992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.763770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92aeae-e67a-445d-ac74-d2d4162f5969 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.783378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3077/6214] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:47.785091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.785513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.786043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.807277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.808503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.808888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.809259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.810141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.815962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.839103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:47.840697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.853582 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3138/6215] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:47.854813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.871408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3078/6216] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:47.872719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.881816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.885399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df6c78-da0f-4613-b211-778162dfae22 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.897905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.899033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.899324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.899598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.900244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.901467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df244e-4700-42ac-9eb4-81e6152a27a9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.902411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0fc44-a097-4a86-a288-32cf679558ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.914699 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3139/6217] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:47.916027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.916326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.916590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.922235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.923023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.924023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.924546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.924919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.926719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:47.943060 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3079/6218] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:47.943345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:47.944158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.944781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:47.956635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12879-2a25-4053-b154-096ec5b7d6e0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.958530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3140/6219] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:47.959561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:47.963158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.963988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.964371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.964750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] PATH_INFO: `/v3/projects/76ee3a51a27d4fedb56c4b7278042a9e/users/23eb9838638c4da992fcc71dad61963a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.967671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:47.973255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f120b6-fe93-4517-bc4e-74a281c6f90c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:47.979129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:47.980782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:47.981070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:47.981554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] PATH_INFO: `/v3/projects/ba9007e00fac49c7a50b5fd15572d276/users/645cbc3ddcde476d89433ff04b5feb5e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:47.991067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ee3a51a27d4fedb56c4b7278042a9e, user_id=23eb9838638c4da992fcc71dad61963a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:47.994823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.002470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9007e00fac49c7a50b5fd15572d276, user_id=645cbc3ddcde476d89433ff04b5feb5e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.005513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d8b3a-acfd-4579-976e-9a23b0706bc1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.024900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3080/6220] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:47 2022] PUT /identity/v3/projects/76ee3a51a27d4fedb56c4b7278042a9e/users/23eb9838638c4da992fcc71dad61963a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:48.025579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.028866 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3141/6221] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:47 2022] PUT /identity/v3/projects/ba9007e00fac49c7a50b5fd15572d276/users/645cbc3ddcde476d89433ff04b5feb5e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:48.034226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e191-c9e6-4e41-817d-2ce75a661dd0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.034913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e191-c9e6-4e41-817d-2ce75a661dd0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.035411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1e191-c9e6-4e41-817d-2ce75a661dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.091291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6db1e191-c9e6-4e41-817d-2ce75a661dd0 None None] MFA Rules not processed for user `23eb9838638c4da992fcc71dad61963a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:48.093094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6db1e191-c9e6-4e41-817d-2ce75a661dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.133326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72ae2d-bc2e-4677-ac6e-d1ea74f3a585 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.170484 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3142/6222] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:48.176616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.176906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.177613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.200761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.201356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.201661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.201909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.202702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:48.202967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.205310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.212632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] MFA Rules not processed for user `645cbc3ddcde476d89433ff04b5feb5e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:48.214726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.224060 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3081/6223] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:48.224060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.226469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.278877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:48.306568 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3143/6224] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:48.306568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.325989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f764d-3c41-40b5-a7a2-d8c87e9f9caa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.326167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca86e3-8e75-4052-937f-0b479564767b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.330414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.331146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.331448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.331595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.331699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.332054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.332335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.332610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.333237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.335178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.343769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.345861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.357216 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3082/6225] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:48.359493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.361491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.400642 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3144/6226] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:48 2022] POST /identity/v3/projects => generated 353 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:48.402406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.443210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:48.484219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad678140-0498-4654-bef2-c60bf305c745 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.554078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfa1cc-118c-40bc-9554-5e472a53917d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.559669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.561098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.561098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.561098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.562781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.577466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.616555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.617342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.618064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.618621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.655589 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3145/6227] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:48 2022] POST /identity/v3/users => generated 277 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:48.656505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.661399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.691506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0b1d7-449d-4cce-8647-7c94e4952a10 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.703105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:48.705847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:48.706336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:48.706645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] PATH_INFO: `/v3/projects/61733d1dfced469c82333e9e63905bf0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:48.711682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61733d1dfced469c82333e9e63905bf0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.717183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.743492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:48.745067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:48.763510 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3083/6228] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:48.764375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.766242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:48.850951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61733d1dfced469c82333e9e63905bf0', 'request_id': 'req-56afd67c-9488-4ca3-b115-812dd14980f0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:48.873730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61733d1dfced469c82333e9e63905bf0', 'request_id': 'req-56afd67c-9488-4ca3-b115-812dd14980f0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:48.905087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61733d1dfced469c82333e9e63905bf0', 'request_id': 'req-56afd67c-9488-4ca3-b115-812dd14980f0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:48.948053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] The token cache is being invalidate because project 61733d1dfced469c82333e9e63905bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:48.948455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:48.955575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:48.961396 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3146/6229] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:48 2022] DELETE /identity/v3/projects/61733d1dfced469c82333e9e63905bf0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:48.962336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:48.969815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a13795-7d3a-454f-baa8-b62ac9cc8e04 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:48.971081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.064173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:49.079106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:49.100851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd67c-9488-4ca3-b115-812dd14980f0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.109273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.109907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.110192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.110512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.111837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.128196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95389c57-8195-484f-a73d-d9b62d68b310 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.154111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.156203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.156451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3147/6230] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:49.156952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.159094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.159753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127e5e-7595-47a2-9c35-deea64b17975 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.160066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127e5e-7595-47a2-9c35-deea64b17975 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.160444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de127e5e-7595-47a2-9c35-deea64b17975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.166928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.204383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.212184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.215362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-de127e5e-7595-47a2-9c35-deea64b17975 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:49.219610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-de127e5e-7595-47a2-9c35-deea64b17975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.236190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3084/6231] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:49.237725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.249940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f626-be59-41e2-b95c-17f71372a1da service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.252384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3148/6232] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:49.261463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.261796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.262216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.282624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.283695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.284075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.284702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.285453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.290448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.300561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:49.302198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.313236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3149/6233] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:49.318197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.329829 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3085/6234] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:49.331630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.346153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e38d2-939e-4ca8-aa1c-fe03959c056c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.351443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb5562-e3b8-44dd-8515-cc94d7721c30 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.355923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.356590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.356807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.357075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.357722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:49.358020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.362785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.365684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.366362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.366667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.366942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] PATH_INFO: `/v3/projects/a903455b85b443c9b55578a68ad9f8ff/users/8f42624017d54281b414c4a4f7baf977/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.396003 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3150/6235] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:49.397116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.406409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.410734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a903455b85b443c9b55578a68ad9f8ff, user_id=8f42624017d54281b414c4a4f7baf977, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.413411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.422948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9027-5c17-440d-8a6a-11641dee9b86 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.432237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.432967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.433382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.433722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] PATH_INFO: `/v3/users/4cd6575e42b0433a9d5be95b34bc4a8f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.435359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd6575e42b0433a9d5be95b34bc4a8f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.437623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.442535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3086/6236] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:49 2022] PUT /identity/v3/projects/a903455b85b443c9b55578a68ad9f8ff/users/8f42624017d54281b414c4a4f7baf977/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:49.443193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.543528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd6575e42b0433a9d5be95b34bc4a8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:49.545120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd6575e42b0433a9d5be95b34bc4a8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:49.563775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd6575e42b0433a9d5be95b34bc4a8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:49.578910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f915103-2e07-4611-b471-a7d4ea0bce7f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.586890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.587529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.587804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.587927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd6575e42b0433a9d5be95b34bc4a8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:49.588122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.588692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.588814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1fe8b446-254e-4dd0-95e9-a028a0f5b9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd6575e42b0433a9d5be95b34bc4a8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:49.605187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3151/6237] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:49 2022] DELETE /identity/v3/users/4cd6575e42b0433a9d5be95b34bc4a8f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:49.607685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53addb1-b94f-42c0-95d8-5fdc7dbac79c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.607920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.608214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.608493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.662095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] MFA Rules not processed for user `8f42624017d54281b414c4a4f7baf977`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:49.663518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.670706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3087/6238] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/projects => generated 389 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:49.672092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.672378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.672653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.716219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:49.717227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.791875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3152/6239] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:49.792941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.794561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.816801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:49.828650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3088/6240] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:49.831213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.832717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.899299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:49.906625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21407df0-7183-4b5e-a39e-11d4d06a8710 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.916296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.917064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.917352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.917752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.918399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:49.920669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:49.948042 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3153/6241] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:49 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:49.948816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:49.951412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:49.968615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78eba-716f-43db-b0a1-5ccda384290a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.969584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6ca22-7e46-4d99-8a20-845f2d01de55 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:49.975413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.976776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.977212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.979181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:49.986404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:49.987216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:49.987703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:49.987991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.007265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.009107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.023129 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3154/6242] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:50.023919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.048973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88ac47-474c-488e-89c6-b30ca0efe919 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.067380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.069650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.070024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.070421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.071171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.073488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.095712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.098363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.104417 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3155/6243] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:50.108795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.112167 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3089/6244] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:50.128748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847bf9-be12-4885-8fd8-bf9433ac2844 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.136036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.136714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.136998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.137270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.137907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.139800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f3cd7-e606-47ba-9ade-88ecbf73a5d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.165121 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3156/6245] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:50.166486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.179807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eb1a2-1030-4323-9c9b-1636eb76fd0f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.189512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.191414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.191681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.191969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] PATH_INFO: `/v3/projects/9608e3e5880f47d0abdf8ae5070c7803/users/15065d95073c41da999d6edb86b9fd9c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.225069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9608e3e5880f47d0abdf8ae5070c7803, user_id=15065d95073c41da999d6edb86b9fd9c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.230379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.256122 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3090/6246] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:50 2022] PUT /identity/v3/projects/9608e3e5880f47d0abdf8ae5070c7803/users/15065d95073c41da999d6edb86b9fd9c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:50.270831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.270831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.273061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.316244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] MFA Rules not processed for user `15065d95073c41da999d6edb86b9fd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:50.319566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:50.369351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.371360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:50.436315 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3157/6247] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:50.475559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.515606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3234c-dfe2-4cd4-ad99-461f4c2ca417 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.523056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.523923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.524197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.524476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] PATH_INFO: `/v3/users/23eb9838638c4da992fcc71dad61963a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.526414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23eb9838638c4da992fcc71dad61963a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.528201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.596913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18793a-71ce-4524-b8e1-f2558ba8609e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.603158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.604037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.604615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.605054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.610093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23eb9838638c4da992fcc71dad61963a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:50.610990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23eb9838638c4da992fcc71dad61963a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:50.617270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:50.621196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.623213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990924a-a852-4b2a-9b11-3aaa46056100 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:50.628811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:50.634653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23eb9838638c4da992fcc71dad61963a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:50.645332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3158/6248] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:50.653962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23eb9838638c4da992fcc71dad61963a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:50.661632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23eb9838638c4da992fcc71dad61963a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:50.706517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3091/6249] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:50 2022] DELETE /identity/v3/users/23eb9838638c4da992fcc71dad61963a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:50.706735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.706863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.707139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.743698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:50.745143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:50.774224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.776036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:50.899872 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3159/6250] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:12:50.931791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754bb4d-4923-46a3-b617-4b184ba9a76a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:50.940482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:50.941840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:50.942412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:50.943502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] PATH_INFO: `/v3/users/15065d95073c41da999d6edb86b9fd9c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:50.944217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:50.945774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:50.946835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15065d95073c41da999d6edb86b9fd9c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:50.949445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:51.040945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e57985-d97c-4917-ba7b-69ecf0a0ebf6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:51.045842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:51.046408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:51.046783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:51.047543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:51.052515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15065d95073c41da999d6edb86b9fd9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.054899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15065d95073c41da999d6edb86b9fd9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.059934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:51.061480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3dfdf6-e4ff-4bf2-9c14-dc34bda7ba85 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:51.075058 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3160/6251] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:12:51.078202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15065d95073c41da999d6edb86b9fd9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.087607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15065d95073c41da999d6edb86b9fd9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.088620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15065d95073c41da999d6edb86b9fd9c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.124675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3092/6252] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:50 2022] DELETE /identity/v3/users/15065d95073c41da999d6edb86b9fd9c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:51.132743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:51.133189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:51.133446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:51.157388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:51.158665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:51.236433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3161/6253] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:51.253060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:51.257262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:51.350582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:51.359346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735575e-3ad2-4432-96d3-c396ae12edc6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:51.367856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:51.369507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:51.370133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:51.370759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:51.389637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:51.391492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:51.408759 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3093/6254] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:51.639664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:51.641265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:51.669650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d9722-68f6-4e56-91bc-edd497ebbbd4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:51.690836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:51.691594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:51.692820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:51.692820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] PATH_INFO: `/v3/projects/ac17de168d364cfca453c2eec2529ac2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:51.693641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac17de168d364cfca453c2eec2529ac2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:51.696120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:51.781359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:51.809581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac17de168d364cfca453c2eec2529ac2', 'request_id': 'req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.822262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac17de168d364cfca453c2eec2529ac2', 'request_id': 'req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.832166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac17de168d364cfca453c2eec2529ac2', 'request_id': 'req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.854936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] The token cache is being invalidate because project ac17de168d364cfca453c2eec2529ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:51.855622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.856519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:51.859274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3162/6255] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:51 2022] DELETE /identity/v3/projects/ac17de168d364cfca453c2eec2529ac2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:51.864483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:51.877797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f46f7f-3fda-43b5-97aa-eb7b74289910 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:52.041952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:52.042658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:52.042948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:52.043228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] PATH_INFO: `/v3/users/1d2bb0bc7aa7443e85dcc7570f224141` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:52.044849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d2bb0bc7aa7443e85dcc7570f224141)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:52.047523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:52.183994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d2bb0bc7aa7443e85dcc7570f224141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:52.191156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d2bb0bc7aa7443e85dcc7570f224141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:52.236148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d2bb0bc7aa7443e85dcc7570f224141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:52.246724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d2bb0bc7aa7443e85dcc7570f224141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:52.249435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d2bb0bc7aa7443e85dcc7570f224141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:52.313361 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3094/6256] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:51 2022] DELETE /identity/v3/users/1d2bb0bc7aa7443e85dcc7570f224141 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:52.804030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:52.806873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:52.833571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93196-a45d-45bb-b02c-ec3a7c26d7d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:52.992261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:52.992919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:52.993472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:52.993753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] PATH_INFO: `/v3/projects/9608e3e5880f47d0abdf8ae5070c7803` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:52.995901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9608e3e5880f47d0abdf8ae5070c7803)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:52.999661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:53.103371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.111597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:53.142726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c731fe-d747-43ae-9a2d-6c060cff390a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:53.154845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:53.156685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.157077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.159404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] PATH_INFO: `/v3/projects/76ee3a51a27d4fedb56c4b7278042a9e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.162006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ee3a51a27d4fedb56c4b7278042a9e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:53.164012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:53.278647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9608e3e5880f47d0abdf8ae5070c7803', 'request_id': 'req-a09c2ae4-90c9-46a7-b360-da54fba47bb7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.309442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9608e3e5880f47d0abdf8ae5070c7803', 'request_id': 'req-a09c2ae4-90c9-46a7-b360-da54fba47bb7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.328210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9608e3e5880f47d0abdf8ae5070c7803', 'request_id': 'req-a09c2ae4-90c9-46a7-b360-da54fba47bb7'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.352754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ee3a51a27d4fedb56c4b7278042a9e', 'request_id': 'req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.365592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ee3a51a27d4fedb56c4b7278042a9e', 'request_id': 'req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.367362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] The token cache is being invalidate because project 9608e3e5880f47d0abdf8ae5070c7803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:53.367842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.370198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a09c2ae4-90c9-46a7-b360-da54fba47bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.373148 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3163/6257] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:52 2022] DELETE /identity/v3/projects/9608e3e5880f47d0abdf8ae5070c7803 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:53.380622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cae5b4-61e8-4351-b44e-83664726f0be None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.381178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cae5b4-61e8-4351-b44e-83664726f0be None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.382806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cae5b4-61e8-4351-b44e-83664726f0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.406654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ee3a51a27d4fedb56c4b7278042a9e', 'request_id': 'req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.415741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:53.432736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b3cae5b4-61e8-4351-b44e-83664726f0be None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:53.434322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3cae5b4-61e8-4351-b44e-83664726f0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:53.450131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] The token cache is being invalidate because project 76ee3a51a27d4fedb56c4b7278042a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:53.450620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.451566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fa4ca18-58dd-42fd-9095-0c54ebf3b304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:53.456256 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3095/6258] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:53 2022] DELETE /identity/v3/projects/76ee3a51a27d4fedb56c4b7278042a9e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:53.465578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d9815-039b-40fc-9633-5e38a845ea67 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.466007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d9815-039b-40fc-9633-5e38a845ea67 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.466327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d9815-039b-40fc-9633-5e38a845ea67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.527755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-997d9815-039b-40fc-9633-5e38a845ea67 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:53.529012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-997d9815-039b-40fc-9633-5e38a845ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:53.555968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:53.575105 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3164/6259] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:53.595212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.595746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.596117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.643073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:53.644207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:53.670433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3165/6260] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:53.674429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.695958 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3096/6261] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:53.708298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.708592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.708860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.710426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b972a4-b34a-4de7-a8fc-4c6a4b3dfa53 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:53.716957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:53.717587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.717854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.718127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.718840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:53.719512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:53.724747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:53.751666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:53.753921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:53.765361 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3166/6262] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:12:53.778957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.791123 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3097/6263] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:53.792397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.801143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ddd98-37ce-4ed2-8a8a-6904fbc0777d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:53.808180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:53.809990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.810573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.811017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.811823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:53.815063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:53.819147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f87d2c-4e9f-486a-a149-ff1ad0731188 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:53.831321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:53.832160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:53.832610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:53.833072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:53.833889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:53.834972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:53.836791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:53.856344 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3098/6264] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:53.866767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.873816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3167/6265] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:53.878242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:53.892393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410775a1-5e54-42f2-ac7f-9696787384d1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:53.999351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.000263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.000555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.000833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.004265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.008397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.026977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c8c59-1e8a-4369-bd7d-97a36e7e16f9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.043898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.044811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.046430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.046916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.048312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.057821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.071759 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3099/6266] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/projects => generated 383 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:54.072805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.138950 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3168/6267] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:53 2022] POST /identity/v3/users => generated 296 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:54.142703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.172107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0fbc-3f6f-4daa-be3e-97996b1233e6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.227578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ca056-3dd5-4c9e-b88b-c1f2d7d3463c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.233146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.233951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.234336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.234760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.235703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.237862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.249783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:54.295971 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3100/6268] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:54 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:54.296721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.302606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:54.331666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154623-4cf0-4543-8ef2-6fc8d333b809 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.346367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.347187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.347477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.347749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] PATH_INFO: `/v3/projects/771d8a6abcf1416e970d19968c258f02` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.351405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771d8a6abcf1416e970d19968c258f02)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.353440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.407953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.408590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.408874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.409155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.414609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.417234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.464976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3169/6269] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:54.468593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.468734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771d8a6abcf1416e970d19968c258f02', 'request_id': 'req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.494342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771d8a6abcf1416e970d19968c258f02', 'request_id': 'req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.508038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db0f8-4380-4c54-9cea-3e2fdc2f506b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.518455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771d8a6abcf1416e970d19968c258f02', 'request_id': 'req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.539393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] The token cache is being invalidate because project 771d8a6abcf1416e970d19968c258f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:54.539640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.540370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.545217 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3101/6270] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:54 2022] DELETE /identity/v3/projects/771d8a6abcf1416e970d19968c258f02 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:54.546250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.548218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:54.675595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:54.693718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65912825-1c89-4c4d-b1a4-f0d8a06d13b5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.718369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.720000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.720265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.722684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.723417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.725881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.726861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.727589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.728531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.729533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.731342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.736832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.760548 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3102/6271] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:54.761229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.771407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:54.772274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3170/6272] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:54.774224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.796393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3e0eb-c8e5-4873-b532-8e3d307d4c11 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.800984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61861a-5f0b-469f-9e1a-ef4a12dceca5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:54.805895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.807130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.807484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.808282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] PATH_INFO: `/v3/users/447fe4899e8b425b8db7c70d8f920a91` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.810306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447fe4899e8b425b8db7c70d8f920a91)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.815246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.820717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:54.821405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:54.821687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:54.821968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] PATH_INFO: `/v3/projects/15d9beb81b534df9884cb6fea9a85f3f/users/e331bee95b554f7d8d18118ef8f2333b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:54.864741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d9beb81b534df9884cb6fea9a85f3f, user_id=e331bee95b554f7d8d18118ef8f2333b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:54.868107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:54.897831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3171/6273] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:54 2022] PUT /identity/v3/projects/15d9beb81b534df9884cb6fea9a85f3f/users/e331bee95b554f7d8d18118ef8f2333b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:54.906764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:54.906764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:54.913899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447fe4899e8b425b8db7c70d8f920a91'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.916312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447fe4899e8b425b8db7c70d8f920a91'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.937943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447fe4899e8b425b8db7c70d8f920a91'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.970982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447fe4899e8b425b8db7c70d8f920a91'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.971810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-81faa45a-08db-44bd-9566-0a59de936a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447fe4899e8b425b8db7c70d8f920a91'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:54.974761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=5,keystone:SELECT=80,keystone:DELETE=15 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:55.002073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3103/6274] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:54 2022] DELETE /identity/v3/users/447fe4899e8b425b8db7c70d8f920a91 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:55.002073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.002073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.002073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.042565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] MFA Rules not processed for user `e331bee95b554f7d8d18118ef8f2333b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:55.044247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.062695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bab517-0ac9-4af7-b718-cb5a0c9ab7ac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:55.211716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:55.212309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.212600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.212877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.213532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:55.215395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:55.238456 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3172/6275] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:55.246280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:55.250162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.250590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3104/6276] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:55.276906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:55.279600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742737d4-54af-47fd-91d8-6b9e577e004e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:55.280305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.301332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:55.302024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.302307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.304691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] PATH_INFO: `/v3/projects/2e7563fe5cfe4b5ba1783d9ec26d5009/users/f9bee7e327e2490eaa445ec2f61a617b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.339956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7563fe5cfe4b5ba1783d9ec26d5009, user_id=f9bee7e327e2490eaa445ec2f61a617b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:55.343221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01f2cc-54fb-42bd-b601-a31e91fe82ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:55.378551 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3173/6277] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:55 2022] PUT /identity/v3/projects/2e7563fe5cfe4b5ba1783d9ec26d5009/users/f9bee7e327e2490eaa445ec2f61a617b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:55.383417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.384071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.384514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.444738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:55.446166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.495258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:55.541739 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3174/6278] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:55.541964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c94660-68b1-4c17-ae7e-ce3b02f3ad9d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:55.543610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:55.546617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.683858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a691725-9a38-480e-8bc3-ef669607e43f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:55.693674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:55.694305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.694612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.694884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.698952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:55.700417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.700941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.701522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.706518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:55.748081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:55.757030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:55.783979 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3175/6279] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:55.785233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:55.808311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd297-9d06-4db4-8ac3-5a6f00742452 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:55.809366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:55.821585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:55.823089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:55.823601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:55.824128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:55.851078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:55.853222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:55.855510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:55.857069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffead0cc-5a9a-4e6e-bc1b-b2f6378854cf service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:55.878807 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3176/6280] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:55.889199 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3105/6281] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:57.361015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:57.362791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:57.384690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2438a-25d7-4b17-94d3-03ea60a5b2c3 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:57.397209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:57.398072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:57.398436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:57.398861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] PATH_INFO: `/v3/users/8f42624017d54281b414c4a4f7baf977` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:57.400579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f42624017d54281b414c4a4f7baf977)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:57.403918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:57.488590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f42624017d54281b414c4a4f7baf977'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:57.489479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f42624017d54281b414c4a4f7baf977'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:57.499446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f42624017d54281b414c4a4f7baf977'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:57.542919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f42624017d54281b414c4a4f7baf977'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:57.544341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f42624017d54281b414c4a4f7baf977'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:57.577997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3177/6282] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:12:57 2022] DELETE /identity/v3/users/8f42624017d54281b414c4a4f7baf977 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:57.582698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:57.583540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:57.584002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:57.613370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:57.627772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:57.770035 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3106/6283] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:57.811081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:57.827399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:57.913366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:57.982811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:57.984520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:58.028760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ae494-2419-4a53-9a84-fc2c9e7b2180 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.043084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.044027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.045855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.046314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] PATH_INFO: `/v3/projects/57256c04221643029b91ab7a3aa82819` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.051188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31035389-c5fb-4262-994c-fb3dd6e4316a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.051697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57256c04221643029b91ab7a3aa82819)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.053967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.056465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.057314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.057909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.058121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.070047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.072718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ff115-fd27-4a1f-8978-643dd6070f0c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.086614 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3178/6284] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:12:58.135696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57256c04221643029b91ab7a3aa82819', 'request_id': 'req-9b494428-503d-4467-9321-2bc717facf69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:58.143524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57256c04221643029b91ab7a3aa82819', 'request_id': 'req-9b494428-503d-4467-9321-2bc717facf69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:58.154947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57256c04221643029b91ab7a3aa82819', 'request_id': 'req-9b494428-503d-4467-9321-2bc717facf69'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:58.179131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] The token cache is being invalidate because project 57256c04221643029b91ab7a3aa82819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:58.179494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:58.180308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9b494428-503d-4467-9321-2bc717facf69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:58.187391 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3107/6285] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:57 2022] DELETE /identity/v3/projects/57256c04221643029b91ab7a3aa82819 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:58.195455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.195731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.196012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.242659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:58.243965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:58.354548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3179/6286] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:58.362876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.363244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.363426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.409487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:58.410416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:58.423399 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3108/6287] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:58.424863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.453469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6786a80-b14e-456c-8987-fcee28948231 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.465671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.466805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.467317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.467787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.474729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:58.475360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.477498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.510406 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3180/6288] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:58.517179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.529463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06791679-d8c3-4ae1-a9fd-9487093a2562 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.542607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.546394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.546951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.547728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.548859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.551860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.614536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3109/6289] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:58 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:58.618490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.633150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a452ea4-f083-4f31-aa58-a7505cc1aad2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.749360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.749985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.750262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.750556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.752055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.754057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.821071 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3181/6290] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:58 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:12:58.822968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.840884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9bf3b-2341-4410-89e9-213f4aee4c1b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.850359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.852685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.853113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.853467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.854151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.858536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.885789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.886587 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3110/6291] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:58.892029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.892946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:58.910345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac7b4-5029-4763-97a9-7a291b95fb2b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.917181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa2943-afd8-4870-998a-c61a816b3baa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.923402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.924462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.925105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.925551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.929748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.931979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.940309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.940977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.941256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.941526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] PATH_INFO: `/v3/projects/a903455b85b443c9b55578a68ad9f8ff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:58.943595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a903455b85b443c9b55578a68ad9f8ff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:58.945557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:58.947484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3111/6292] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:58.952106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:58.968309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e50c3-0d2a-4076-97fd-15e018da9b09 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:58.990911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:58.991616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:58.992082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:58.992374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] PATH_INFO: `/v3/projects/026ea39b3679444a9c3541e58d2e9f7b/users/c1acf8d4b1384c8c8d7014bd8baf897e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.022448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:59.038730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=026ea39b3679444a9c3541e58d2e9f7b, user_id=c1acf8d4b1384c8c8d7014bd8baf897e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.046616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3c0aa-15ca-493f-bb92-16ea56fcc26f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.051094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:59.086473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a903455b85b443c9b55578a68ad9f8ff', 'request_id': 'req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:59.088376 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3112/6293] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:58 2022] PUT /identity/v3/projects/026ea39b3679444a9c3541e58d2e9f7b/users/c1acf8d4b1384c8c8d7014bd8baf897e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:12:59.102870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.103158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.103437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.115583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a903455b85b443c9b55578a68ad9f8ff', 'request_id': 'req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:59.133077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a903455b85b443c9b55578a68ad9f8ff', 'request_id': 'req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:59.165428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] The token cache is being invalidate because project a903455b85b443c9b55578a68ad9f8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:12:59.166127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:59.167506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ce1b0e55-a9b4-4381-aa3a-2d7b00a3a3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:12:59.171366 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3182/6294] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:12:58 2022] DELETE /identity/v3/projects/a903455b85b443c9b55578a68ad9f8ff => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:12:59.177955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f1f-3a5c-4eb6-ba82-9f713301db88 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.178864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f1f-3a5c-4eb6-ba82-9f713301db88 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.179817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba545f1f-3a5c-4eb6-ba82-9f713301db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.189319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] MFA Rules not processed for user `c1acf8d4b1384c8c8d7014bd8baf897e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:59.195344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:59.244555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ba545f1f-3a5c-4eb6-ba82-9f713301db88 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:59.246984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ba545f1f-3a5c-4eb6-ba82-9f713301db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:59.387664 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3113/6295] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:59.392359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.394086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:59.398748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3183/6296] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:12:59.406464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.406772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.407838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.425595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bffe5-6745-4c6d-8502-2130fa964542 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.435685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.436879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.437351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.437781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.439431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.442037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.443487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:12:59.444900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:12:59.465519 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3184/6297] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:12:59.472701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.487333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab595cc2-2706-4d8d-8c7b-01748c422592 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.494155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.494952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.496078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.496602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.497628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:12:59.498039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.499306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3114/6298] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:59.501678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.504316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.519031 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3185/6299] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:12:59.523228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.538372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38544-34cb-4fd4-861a-ba6f3b4a23bb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.678738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.683481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.683987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.684484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.685267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502fb6-7b5a-4197-973d-146d0eb77d5f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.685877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.688342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.694726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.695442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.695791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.696125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.696835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.701897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.747953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3115/6300] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:59.749325 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3186/6301] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:12:59.754062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.755567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.775314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cf506-13c7-4bd8-a600-17f017902a02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.821225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:59.883873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51100a3-7967-459d-9f98-ee6042430002 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.895167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:12:59.897839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.898470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.899400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.899669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.900402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.908445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.930249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.931169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.931391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.931760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.932509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:12:59.934757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:12:59.952951 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3116/6302] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:12:59.954934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.958988 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3187/6303] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:12:59 2022] POST /identity/v3/users => generated 285 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:12:59.963200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:12:59.968173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee70ba-f2db-421b-a3ad-a984a230011f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.975619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.976584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:12:59.977021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:12:59.977485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] PATH_INFO: `/v3/projects/44a13cc297a446208816af67a5502a15/users/5c5d0cac25164ecba60f1fc493407026/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:12:59.984109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac637-5941-4fec-ad0b-dfb909850ffe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:12:59.996889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:12:59.997819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.001499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a13cc297a446208816af67a5502a15, user_id=5c5d0cac25164ecba60f1fc493407026, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.001865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.002320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.003645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.006881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.008987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.024534 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3188/6304] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:00.028603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.044408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3117/6305] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:12:59 2022] PUT /identity/v3/projects/44a13cc297a446208816af67a5502a15/users/5c5d0cac25164ecba60f1fc493407026/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:00.049490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.070946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d1fd3-3781-4bc1-a43b-d65fa7025950 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.208830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:00.209450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.209728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.210003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.210679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.214026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.247505 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3189/6306] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:00.249535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.255161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded494-0155-406a-a43b-32e9cb5d3e4a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.269611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:00.270629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.270911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.271183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.271823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.275355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4238c1-7aad-41ed-9593-783a4ac472bf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.276410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.288449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:00.289262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.289407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.289661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] PATH_INFO: `/v3/projects/258fe08c46fc4d42b0d66ded69d587fa/users/6f507a26260d491cbef6c09a15e1ee6b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.296597 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3118/6307] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:00.304080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.327595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] RBAC: Authorizing `identity:create_grant(project_id=258fe08c46fc4d42b0d66ded69d587fa, user_id=6f507a26260d491cbef6c09a15e1ee6b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.331472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162eb1f-b64e-499e-8491-f8e58855769e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.331689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74217cc0-e466-4922-8eef-a45c0cbae31b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.341306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:00.341995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.342386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.342696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] PATH_INFO: `/v3/projects/44a13cc297a446208816af67a5502a15/users/5c5d0cac25164ecba60f1fc493407026/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.345365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a13cc297a446208816af67a5502a15, user_id=5c5d0cac25164ecba60f1fc493407026, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.351648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a64ba-cf5d-453f-ad72-93d8d68c14fa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.360353 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3190/6308] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:00 2022] PUT /identity/v3/projects/258fe08c46fc4d42b0d66ded69d587fa/users/6f507a26260d491cbef6c09a15e1ee6b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:00.373235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.373532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.373784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.395550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3119/6309] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:00 2022] PUT /identity/v3/projects/44a13cc297a446208816af67a5502a15/users/5c5d0cac25164ecba60f1fc493407026/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:00.406417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.406706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.407104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.435542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] MFA Rules not processed for user `6f507a26260d491cbef6c09a15e1ee6b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:00.437348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:00.477013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] MFA Rules not processed for user `5c5d0cac25164ecba60f1fc493407026`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:00.478375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:00.546948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3191/6310] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:00.548731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.573376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:00.655208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3120/6311] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:00.695616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:00.698202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:00.747954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55668d12-10e5-4d25-baa5-50cb10f91885 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.768730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:00.769369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:00.769650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:00.769920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:00.770588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:00.772517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:00.783005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:00.845893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c956771-209e-4736-9518-cc28ea5e11b2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:00.856986 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3192/6312] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:00 2022] POST /identity/v3/projects => generated 369 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:00.861884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.000573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.001449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.001775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.002046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.020666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a0ad-b079-40c8-b872-0b4b3630d72c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.028947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.030379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.032301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.033050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.034109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.036432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.092501 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3193/6313] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:00 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:01.095416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.114633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812606-a822-4d57-8f01-cc3b233360b5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.133159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.133905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.134168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.134442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.135110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.136733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.144357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.146434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.154798 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3194/6314] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:01.158725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.169790 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3121/6315] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:01.174299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54d3bd-84d9-4ad8-aab1-69c27a258f11 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.183867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.184616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.184900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.185172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] PATH_INFO: `/v3/projects/7165924e149d47c4bf426acfe6030a12/users/2a39942743aa468c8ad4c2d88006ccf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.206679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7165924e149d47c4bf426acfe6030a12, user_id=2a39942743aa468c8ad4c2d88006ccf2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.211213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.237332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3195/6316] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:01 2022] PUT /identity/v3/projects/7165924e149d47c4bf426acfe6030a12/users/2a39942743aa468c8ad4c2d88006ccf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:01.241135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.258295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25900-068a-4f8e-a1ac-c12927a198d1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.425198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.425731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.426007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.426278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.427498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.429498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311561c1-064b-4a29-b4cb-2ce698a97102 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.470461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3122/6317] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:13:01.472544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.493995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50162369-12bf-4c6e-be7e-e02efa62880d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.511661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.512468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.513563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.513958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] PATH_INFO: `/v3/projects/7165924e149d47c4bf426acfe6030a12/users/2a39942743aa468c8ad4c2d88006ccf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.521159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7165924e149d47c4bf426acfe6030a12, user_id=2a39942743aa468c8ad4c2d88006ccf2, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.530422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:01.586790 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3196/6318] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:01 2022] PUT /identity/v3/projects/7165924e149d47c4bf426acfe6030a12/users/2a39942743aa468c8ad4c2d88006ccf2/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:01.589581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.593816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.595568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.679721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] MFA Rules not processed for user `2a39942743aa468c8ad4c2d88006ccf2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:01.680911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:01.713539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:01.779642 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3123/6319] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:01.786312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:01.789488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:01.882047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:01.920403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8dc451-5c72-4d79-8a7f-dcf54b07b8c6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:01.950748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:01.950748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:01.950748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:01.950748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:01.977166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:01.978846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:02.006391 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3197/6320] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:02.091585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:02.115415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e15bd1-1db0-4d0f-a1fa-9ad5734f94da None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:02.132865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:02.133740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:02.134113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:02.134735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:02.301165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:02.303194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcc756-05b4-46b8-a341-fea0ab66572d service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:02.319135 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3124/6321] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:06.541553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:06.656083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b50160-b1af-4f81-8982-1983a185b767 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:06.678758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:06.679747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:06.680212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:06.680664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] PATH_INFO: `/v3/users/6f507a26260d491cbef6c09a15e1ee6b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:06.683015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f507a26260d491cbef6c09a15e1ee6b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:06.685204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:06.827601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f507a26260d491cbef6c09a15e1ee6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:06.829347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f507a26260d491cbef6c09a15e1ee6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:06.856298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f507a26260d491cbef6c09a15e1ee6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:06.879398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f507a26260d491cbef6c09a15e1ee6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:06.880957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f507a26260d491cbef6c09a15e1ee6b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:06.911422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3198/6322] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:06 2022] DELETE /identity/v3/users/6f507a26260d491cbef6c09a15e1ee6b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:06.915615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:06.916540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:06.917054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:06.940602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:06.942430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:07.044417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3125/6323] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:07.065961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:07.067944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:07.149141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f164b-7021-41e9-baf4-5c16601dac22 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:07.156010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:07.157580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:07.158399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:07.158796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:07.176118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:07.177888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:07.185584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:07.194249 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3199/6324] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:07.420843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:07.423457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:07.461107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e76a69-de0b-4870-aa2a-d3d88731c780 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:07.479127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:07.480309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:07.480309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:07.480551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:07.483317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:07.646308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:07.688417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:07.690169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:07.714040 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3126/6325] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:09.649075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:09.650807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:09.675839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5aea9-55e2-4349-ba65-7499f91f7c25 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:09.694310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:09.695288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:09.695570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:09.697731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] PATH_INFO: `/v3/projects/258fe08c46fc4d42b0d66ded69d587fa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:09.701038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258fe08c46fc4d42b0d66ded69d587fa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:09.706677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:09.864926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258fe08c46fc4d42b0d66ded69d587fa', 'request_id': 'req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:09.891108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258fe08c46fc4d42b0d66ded69d587fa', 'request_id': 'req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:09.923487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258fe08c46fc4d42b0d66ded69d587fa', 'request_id': 'req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:09.935806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] The token cache is being invalidate because project 258fe08c46fc4d42b0d66ded69d587fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:09.936208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:09.937350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:09.940856 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3200/6326] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:09 2022] DELETE /identity/v3/projects/258fe08c46fc4d42b0d66ded69d587fa => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:09.944735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:09.963175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe19510-a6f1-4367-8989-ac6a8ea33d72 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:10.137135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:10.137848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:10.138138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:10.138532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] PATH_INFO: `/v3/users/2a39942743aa468c8ad4c2d88006ccf2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:10.141085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a39942743aa468c8ad4c2d88006ccf2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:10.143351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:10.283645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a39942743aa468c8ad4c2d88006ccf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.287213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a39942743aa468c8ad4c2d88006ccf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.314743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a39942743aa468c8ad4c2d88006ccf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.325175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a39942743aa468c8ad4c2d88006ccf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.327133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-33b7e15f-e78a-4620-9719-6db440b0f058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a39942743aa468c8ad4c2d88006ccf2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.389438 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3127/6327] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:09 2022] DELETE /identity/v3/users/2a39942743aa468c8ad4c2d88006ccf2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:10.598937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:10.600542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:10.626714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6ee74-0875-47f6-8a7f-12cdec5e0418 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:10.756279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:10.756981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:10.757223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:10.757500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] PATH_INFO: `/v3/users/e331bee95b554f7d8d18118ef8f2333b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:10.759156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e331bee95b554f7d8d18118ef8f2333b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:10.761164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:10.872019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e331bee95b554f7d8d18118ef8f2333b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.872730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e331bee95b554f7d8d18118ef8f2333b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.894791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e331bee95b554f7d8d18118ef8f2333b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.923258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e331bee95b554f7d8d18118ef8f2333b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.924693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e331bee95b554f7d8d18118ef8f2333b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:10.945696 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3201/6328] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:10 2022] DELETE /identity/v3/users/e331bee95b554f7d8d18118ef8f2333b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:10.950720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:10.951239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:10.951361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:10.983602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:10.985545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:11.133064 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3128/6329] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:11.262351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:11.264426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:11.272617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:11.396242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8aae-1b1e-4b55-9a26-6e1cf1a0a918 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:11.401890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:11.403011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:11.403526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:11.404295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:11.416534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:11.418757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69151e66-344d-46f1-9aa4-24bc8bd0e762 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:11.435742 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3202/6330] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:12.243517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:12.245559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:12.273659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b8f6c-50cc-4ae6-9915-cf32a5a746b9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:12.294251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:12.295741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.296456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.297690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] PATH_INFO: `/v3/projects/7165924e149d47c4bf426acfe6030a12` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.305280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7165924e149d47c4bf426acfe6030a12)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:12.363715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:12.379867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:12.576214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7165924e149d47c4bf426acfe6030a12', 'request_id': 'req-2279c509-8b74-4768-83f3-62c623ab6527'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:12.593355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7165924e149d47c4bf426acfe6030a12', 'request_id': 'req-2279c509-8b74-4768-83f3-62c623ab6527'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:12.617269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7165924e149d47c4bf426acfe6030a12', 'request_id': 'req-2279c509-8b74-4768-83f3-62c623ab6527'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:12.641241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] The token cache is being invalidate because project 7165924e149d47c4bf426acfe6030a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:12.641957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:12.646031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2279c509-8b74-4768-83f3-62c623ab6527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:12.648723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3129/6331] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:12 2022] DELETE /identity/v3/projects/7165924e149d47c4bf426acfe6030a12 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:12.653612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.654753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.655219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.705724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:12.709536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:12.819274 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3203/6332] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:12.828309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.828880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.829355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.853515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:12.855790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:12.860334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:12.862061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:12.876921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3130/6333] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:12.882949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:12.883160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d17be-2653-4bf5-843d-4b57450989ad None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:12.895475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:12.896153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.896489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.896775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] PATH_INFO: `/v3/projects/15d9beb81b534df9884cb6fea9a85f3f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.902699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d9beb81b534df9884cb6fea9a85f3f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:12.908491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f0147-0f9b-4994-86e3-ad58dc6d8467 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:12.913248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:12.918611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:12.919504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.920074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.920570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.921397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:12.923666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:12.926987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:12.949500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3131/6334] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:12.950426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:12.961609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950115-33eb-4ff1-8f98-3bb0224673e2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:12.969163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:12.970112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:12.970402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:12.970843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:12.971764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:12.974129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.020153 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3132/6335] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:12 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:13.026135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.037535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d9beb81b534df9884cb6fea9a85f3f', 'request_id': 'req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:13.052249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d9beb81b534df9884cb6fea9a85f3f', 'request_id': 'req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:13.068312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d9beb81b534df9884cb6fea9a85f3f', 'request_id': 'req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:13.101469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] The token cache is being invalidate because project 15d9beb81b534df9884cb6fea9a85f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:13.101772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:13.103601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-63e3b3d7-88a7-4ff3-9dac-54225ee5e439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:13.117317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3204/6336] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:12 2022] DELETE /identity/v3/projects/15d9beb81b534df9884cb6fea9a85f3f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:13.122369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0f84-4c17-4460-a632-138fe43e78b3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.122702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0f84-4c17-4460-a632-138fe43e78b3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.127311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac0f84-4c17-4460-a632-138fe43e78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.172690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4aac0f84-4c17-4460-a632-138fe43e78b3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:13.173965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4aac0f84-4c17-4460-a632-138fe43e78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:13.206611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b333363-1f7d-4833-9cdd-500aa2a0a9bf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.226364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.226944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:13.233173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.233545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.233802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.235653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.239111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.276263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3133/6337] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:13.280343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.284191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:13.302202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512318-894c-47c3-b646-3e283da2de64 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.310693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.311876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.312360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.312995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.314967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.316935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.339456 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3134/6338] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:13.347027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3205/6339] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:13.347834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.353364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.353633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.353879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.374188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5742-ebcd-4f4e-a366-0fc30eca04ed admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.394676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:13.396672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:13.413476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.414153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.414564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.414924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.415705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.417766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.418869 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3206/6340] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:13.419805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.435482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28de72-0bbc-473c-9145-2684542dc2aa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.442279 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3135/6341] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:13.443442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.444730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.446247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.446735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.447196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.448013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:13.448928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.451288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.476469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc71ad8-6216-4b79-b486-575e3da7e1e1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.477035 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3207/6342] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:13.481511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.494919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.495717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.496101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.496577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] PATH_INFO: `/v3/projects/d4d507c75d254724a4df8510cc33efd2/users/77686934f9fb40d2808cb07b42fb8856/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.502824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24e058-2ef4-4535-8545-7a0b01525509 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.510133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.511233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.511684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.512631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.513598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.516274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.532870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d507c75d254724a4df8510cc33efd2, user_id=77686934f9fb40d2808cb07b42fb8856, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.535769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64427109-62fc-4dac-87ee-624ea3c7d090 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.543891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:13.569340 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3208/6343] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:13.570592 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3136/6344] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:13 2022] PUT /identity/v3/projects/d4d507c75d254724a4df8510cc33efd2/users/77686934f9fb40d2808cb07b42fb8856/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:13.581207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.581729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.582074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.583565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.624016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] MFA Rules not processed for user `77686934f9fb40d2808cb07b42fb8856`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:13.626678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:13.721050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e5ac0-9255-4341-9d72-8636e4c2669b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.733779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.735164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.735664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.736191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.736951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.739255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.757386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3137/6345] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:13.761864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.779837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8e23c-fe8a-45d6-93d3-bb3c00b67ae6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.781026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3209/6346] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:13:13.783253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.787555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.789278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.789573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.789841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.790520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.793677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.799514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b6bef-a176-432f-9685-436a072be54e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.806274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.806992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.807268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.807537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.809971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.812404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:13.825502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3210/6347] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:13.830383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.846129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd1b7-f54b-4692-8d84-451a42368539 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:13.850342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3138/6348] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:13.853260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:13.988702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:13.991131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:13.991969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:13.993150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:13.993956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:13.997528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.015300 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3211/6349] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:14.020388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.033695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2334e-847e-43c1-9714-c73e0ec27503 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.041064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:14.042978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.043917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.044380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.044828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.045644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.048050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.063670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726487e4-0357-42a6-97be-208470740ee0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.083406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.083406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.083406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.083406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] PATH_INFO: `/v3/projects/bdf9dbda5d6447dda073dcc30c045403/users/8e03d4aced164fd1be95a02b07655809/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.085381 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3139/6350] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:13 2022] POST /identity/v3/users => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:14.086584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.123922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af6c56-460f-438b-bd62-cc637f5de0a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.129817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf9dbda5d6447dda073dcc30c045403, user_id=8e03d4aced164fd1be95a02b07655809, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.134026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.134678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.135005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.135288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.135938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.137333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650dbb0-e032-495a-b112-29346eebaa30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.137799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.162256 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3140/6351] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:14.170697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.188637 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3212/6352] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:14 2022] PUT /identity/v3/projects/bdf9dbda5d6447dda073dcc30c045403/users/8e03d4aced164fd1be95a02b07655809/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:14.196868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e76451-01a6-44cd-80ad-e9e70d6f3eef admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.197916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2152-602c-49eb-a3fb-16c952bfc480 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.199004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2152-602c-49eb-a3fb-16c952bfc480 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.200097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce2152-602c-49eb-a3fb-16c952bfc480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.257813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-88ce2152-602c-49eb-a3fb-16c952bfc480 None None] MFA Rules not processed for user `8e03d4aced164fd1be95a02b07655809`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:14.259091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88ce2152-602c-49eb-a3fb-16c952bfc480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:14.293590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:14.327782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.329036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.329326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.329627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] PATH_INFO: `/v3/projects/b80371f938cf40edb0acc45213bdfea6/users/7b7827144c3046ebad2e69dc25115a60/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.357189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80371f938cf40edb0acc45213bdfea6, user_id=7b7827144c3046ebad2e69dc25115a60, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.361129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.401038 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3213/6353] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:14.405859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.406864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.408994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.410206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3141/6354] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:14 2022] PUT /identity/v3/projects/b80371f938cf40edb0acc45213bdfea6/users/7b7827144c3046ebad2e69dc25115a60/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:14.415642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.505766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] MFA Rules not processed for user `f9bee7e327e2490eaa445ec2f61a617b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:14.507478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:14.583216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef58ab-4132-49e5-818f-4e85d6e39681 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.599275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.599909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.600239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.600559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.601432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.605792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.632496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3142/6355] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:14.633571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.639776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:14.725850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3214/6356] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:14.728607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.755550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21226f2c-4368-4cc8-bc10-e9069bc77123 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.770920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.771741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.772030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.772311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] PATH_INFO: `/v3/projects/b80371f938cf40edb0acc45213bdfea6/users/7b7827144c3046ebad2e69dc25115a60/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.778816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80371f938cf40edb0acc45213bdfea6, user_id=7b7827144c3046ebad2e69dc25115a60, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:14.784598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:14.812207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e837-0a26-4cb1-8100-12642b81c36a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:14.827813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3215/6357] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:14 2022] PUT /identity/v3/projects/b80371f938cf40edb0acc45213bdfea6/users/7b7827144c3046ebad2e69dc25115a60/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:14.827813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:14.829710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:14.939957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:14.946428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:14.948179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:14.948712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:14.949218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:14.986174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f092985-9e1f-474d-8562-7de3e68ed3bb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.003353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.003996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.004282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.004555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.005208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.007194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b71114-1319-4ba5-b2c2-afa71e34c2c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.024177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:15.080874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.082923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.084051 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3216/6358] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:14 2022] POST /identity/v3/projects => generated 381 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:15.086798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.087098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.087379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.107685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3143/6359] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:15.108876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.137527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908ddf1-1b04-4e9c-9e1c-622ca16a9093 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.172252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] MFA Rules not processed for user `7b7827144c3046ebad2e69dc25115a60`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:15.173479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:15.278359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.279432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.279813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.280345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.281113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.283284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.341404 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3144/6360] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:15 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:15.343281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.343927 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3217/6361] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:15.355377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.356824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:15.364099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fec2b-b99d-4a9e-9e41-b6b6fdafcf3e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.378110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.378631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.378944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.379786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.382482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.384237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.411352 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3145/6362] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:15.415189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.440803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ad9d0-55b7-42ef-8b3d-2839eacb1a6a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.454602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.455865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.456230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.457885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] PATH_INFO: `/v3/projects/e64ad41b1eb54003bfef94546de43af2/users/372fe818c768428aac3d9e88432ddd5a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.487751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35fa39-f41a-4938-bad8-b1ca0239f13b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.495434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64ad41b1eb54003bfef94546de43af2, user_id=372fe818c768428aac3d9e88432ddd5a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.503747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.507384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.508029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.508313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.508605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.525673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.527799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.534724 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3146/6363] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:15 2022] PUT /identity/v3/projects/e64ad41b1eb54003bfef94546de43af2/users/372fe818c768428aac3d9e88432ddd5a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:15.538967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.555474 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3218/6364] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:15.679084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bbc36-bc87-415c-af39-811da5a84447 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.693558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.694197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.694625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.694813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.695474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.702818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.729750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.731563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3147/6365] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:13:15.747025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bc9f5-c7ec-4d59-8207-6f7402bf1007 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.758700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.759492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.759783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.760062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] PATH_INFO: `/v3/projects/e64ad41b1eb54003bfef94546de43af2/users/372fe818c768428aac3d9e88432ddd5a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.779841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e64ad41b1eb54003bfef94546de43af2, user_id=372fe818c768428aac3d9e88432ddd5a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.793724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418157ea-c279-4f1b-9cdc-d81cd748e905 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:15.817754 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3219/6366] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:15 2022] PUT /identity/v3/projects/e64ad41b1eb54003bfef94546de43af2/users/372fe818c768428aac3d9e88432ddd5a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:15.825396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:15.841660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a012-b43a-477d-8a7c-af4c5dfb76aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:15.881112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:15.957862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:15.960669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:15.961286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:15.961487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] PATH_INFO: `/v3/users/77686934f9fb40d2808cb07b42fb8856` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:15.963331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=77686934f9fb40d2808cb07b42fb8856)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:15.970255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:16.085583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77686934f9fb40d2808cb07b42fb8856'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:16.086355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77686934f9fb40d2808cb07b42fb8856'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:16.099870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77686934f9fb40d2808cb07b42fb8856'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:16.120780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77686934f9fb40d2808cb07b42fb8856'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:16.121880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77686934f9fb40d2808cb07b42fb8856'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:16.175962 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3148/6367] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:15 2022] DELETE /identity/v3/users/77686934f9fb40d2808cb07b42fb8856 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:16.177918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:16.178302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:16.179066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:16.219923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:16.221571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:16.289540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:16.321612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3220/6368] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:16.341317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:16.345906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:16.484562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e478da4-3f51-4c5d-a3df-286d4091de7e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:16.490352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:16.491672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:16.491859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:16.492138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:16.509921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:16.514479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:16.537094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3149/6369] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:17.682408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:17.684203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:17.706131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a0f4d-bd56-4556-965a-5bbd4e66b9ea None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:17.719664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:17.720625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:17.720917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:17.721334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] PATH_INFO: `/v3/projects/d4d507c75d254724a4df8510cc33efd2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:17.723964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d507c75d254724a4df8510cc33efd2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:17.730182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:17.834275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d507c75d254724a4df8510cc33efd2', 'request_id': 'req-10e29654-5433-4ca2-a8dc-bdf472a2aa45'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:17.845374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d507c75d254724a4df8510cc33efd2', 'request_id': 'req-10e29654-5433-4ca2-a8dc-bdf472a2aa45'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:17.856181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d507c75d254724a4df8510cc33efd2', 'request_id': 'req-10e29654-5433-4ca2-a8dc-bdf472a2aa45'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:17.883525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] The token cache is being invalidate because project d4d507c75d254724a4df8510cc33efd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:17.884080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:17.885167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:17.888066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3221/6370] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:17 2022] DELETE /identity/v3/projects/d4d507c75d254724a4df8510cc33efd2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:17.891937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:17.913776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d16fa-73f8-45e0-99d6-2c669e96f009 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:18.059086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:18.089440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:18.090574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:18.091090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:18.091545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] PATH_INFO: `/v3/users/7b7827144c3046ebad2e69dc25115a60` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:18.093560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b7827144c3046ebad2e69dc25115a60)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:18.097931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:18.237386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b7827144c3046ebad2e69dc25115a60'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:18.238756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b7827144c3046ebad2e69dc25115a60'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:18.255029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b7827144c3046ebad2e69dc25115a60'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:18.276239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b7827144c3046ebad2e69dc25115a60'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:18.283110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99884881-0006-4eb4-8668-f7d64f20b8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b7827144c3046ebad2e69dc25115a60'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:18.359442 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3150/6371] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:17 2022] DELETE /identity/v3/users/7b7827144c3046ebad2e69dc25115a60 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:19.644487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:19.646911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:19.667532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e29654-5433-4ca2-a8dc-bdf472a2aa45 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:19.784484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:19.785523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:19.786331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:19.786820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] PATH_INFO: `/v3/projects/b80371f938cf40edb0acc45213bdfea6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:19.788611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80371f938cf40edb0acc45213bdfea6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:19.791698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:19.865148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80371f938cf40edb0acc45213bdfea6', 'request_id': 'req-67efc53f-1a24-468c-a95d-c275ad1695fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:19.888820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80371f938cf40edb0acc45213bdfea6', 'request_id': 'req-67efc53f-1a24-468c-a95d-c275ad1695fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:19.900378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80371f938cf40edb0acc45213bdfea6', 'request_id': 'req-67efc53f-1a24-468c-a95d-c275ad1695fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:19.919193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:19.927876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] The token cache is being invalidate because project b80371f938cf40edb0acc45213bdfea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:19.928767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:19.929860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-67efc53f-1a24-468c-a95d-c275ad1695fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:19.933781 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3222/6372] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:19 2022] DELETE /identity/v3/projects/b80371f938cf40edb0acc45213bdfea6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:19.938825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:19.939672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:19.940150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:19.977801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:19.982145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:20.139188 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3151/6373] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:20.145592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.146100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.146577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.191579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:20.193066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:20.229825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3223/6374] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:20.234588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.248641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8733e38-cfd6-498b-b61e-14996a7f0a01 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.257167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.258127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.258677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.259280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.260294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:20.260819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.262814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.284009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.290418 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3152/6375] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:20.299730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94f578-fa2d-4937-9772-82b639181314 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.305840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.307925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.308463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.308951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.309961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.312707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.367153 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3224/6376] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:20 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:20.368660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.391592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc697f1-419f-4cd5-8a8f-89b28783a2b4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.459888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:20.489519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.491166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.491921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.493598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.494100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.497660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.537905 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3153/6377] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:20 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:20.538909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.556882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd126c-aa53-4cd9-ae89-3d1210edd884 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.572897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.573533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.574548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.574836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.575496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.577236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.605865 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3225/6378] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:20.607167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.626300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ac8a7-0279-4b09-a40d-05d4b0a66b61 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.647888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.648667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.649044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.649937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.651792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.654928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be8c4b1-e782-4558-98e3-ddb0416b744e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.694662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3154/6379] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:20.695915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:20.719767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e207a4a-70f4-41de-98a1-191a364b8ac6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:20.728312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:20.738107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.738981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.739457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] PATH_INFO: `/v3/projects/cfe816f38de9425bb693ff603d7647f9/users/d0dc97f6f4004aa8b3f649bf6f6056bc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.771059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe816f38de9425bb693ff603d7647f9, user_id=d0dc97f6f4004aa8b3f649bf6f6056bc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:20.774004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:20.793037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3226/6380] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:20 2022] PUT /identity/v3/projects/cfe816f38de9425bb693ff603d7647f9/users/d0dc97f6f4004aa8b3f649bf6f6056bc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:20.803572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43de7c-5450-4a01-8639-8a846ddf67f9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:20.803936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43de7c-5450-4a01-8639-8a846ddf67f9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:20.804412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43de7c-5450-4a01-8639-8a846ddf67f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:20.884907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cf43de7c-5450-4a01-8639-8a846ddf67f9 None None] MFA Rules not processed for user `d0dc97f6f4004aa8b3f649bf6f6056bc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:20.886164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf43de7c-5450-4a01-8639-8a846ddf67f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:20.988290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3155/6381] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:21.002640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:21.005264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:21.109867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f16f49-1f82-4ba6-8dc9-9412c3f09a83 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:21.116449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:21.117119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:21.117549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:21.118130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:21.136102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:21.137488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:21.151161 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3227/6382] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:22.854521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7e27e-0ac3-4c41-8e26-a3a75c8b3f3f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:22.855078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7e27e-0ac3-4c41-8e26-a3a75c8b3f3f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:22.855507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7e27e-0ac3-4c41-8e26-a3a75c8b3f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:22.882174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4a7e27e-0ac3-4c41-8e26-a3a75c8b3f3f None None] MFA Rules not processed for user `c1acf8d4b1384c8c8d7014bd8baf897e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:22.884241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4a7e27e-0ac3-4c41-8e26-a3a75c8b3f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:22.979012 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3156/6383] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:23.509991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:23.702202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8a1d8-3c66-4360-899f-b82c9cdc9c3c service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:23.708547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:23.709206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:23.709490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:23.709768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] PATH_INFO: `/v3/users/d0dc97f6f4004aa8b3f649bf6f6056bc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:23.711222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0dc97f6f4004aa8b3f649bf6f6056bc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:23.713473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:23.774807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:23.787256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0dc97f6f4004aa8b3f649bf6f6056bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:23.788539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0dc97f6f4004aa8b3f649bf6f6056bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:23.806393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0dc97f6f4004aa8b3f649bf6f6056bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:23.829053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0dc97f6f4004aa8b3f649bf6f6056bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:23.830254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0dc97f6f4004aa8b3f649bf6f6056bc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:23.869485 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3228/6384] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:23 2022] DELETE /identity/v3/users/d0dc97f6f4004aa8b3f649bf6f6056bc => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:23.877348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:23.879134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:23.879677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:23.912001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:23.913701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:24.013168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3157/6385] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:24.051006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:24.053322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:24.178245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9592fec-4512-48c8-87ae-a4ad399ccf0a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:24.183406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fded7-9786-46c5-9410-575f542610af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:24.184152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fded7-9786-46c5-9410-575f542610af service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:24.184436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fded7-9786-46c5-9410-575f542610af service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:24.184716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fded7-9786-46c5-9410-575f542610af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:24.202382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fded7-9786-46c5-9410-575f542610af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:24.204721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fded7-9786-46c5-9410-575f542610af service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:24.219634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3229/6386] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:25.537013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:25.538825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:25.553356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:25.558328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026cc8f-bfd7-450c-a34a-c7a41bffaccb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:25.574137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:25.574956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:25.575465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:25.575865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] PATH_INFO: `/v3/projects/cfe816f38de9425bb693ff603d7647f9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:25.579687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe816f38de9425bb693ff603d7647f9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:25.581764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:25.716628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe816f38de9425bb693ff603d7647f9', 'request_id': 'req-24d5c654-c9ac-4bf3-8def-bf6c935efedd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:25.724395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe816f38de9425bb693ff603d7647f9', 'request_id': 'req-24d5c654-c9ac-4bf3-8def-bf6c935efedd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:25.739742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe816f38de9425bb693ff603d7647f9', 'request_id': 'req-24d5c654-c9ac-4bf3-8def-bf6c935efedd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:25.758281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] The token cache is being invalidate because project cfe816f38de9425bb693ff603d7647f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:25.758957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:25.760446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-24d5c654-c9ac-4bf3-8def-bf6c935efedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:25.762878 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3158/6387] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:25 2022] DELETE /identity/v3/projects/cfe816f38de9425bb693ff603d7647f9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:25.778472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:25.779868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:25.781157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:25.829420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:25.831964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:26.005145 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3230/6388] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:26.007395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.007729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.007948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.064853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:26.068436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:26.116609 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3159/6389] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:26.117656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.136335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ff0ba-e7f3-45f4-8c85-2340d3520223 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.145591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.146596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.147031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.147804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.148964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:26.149578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.151524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.172454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.176621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3231/6390] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:26.189266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069369b7-26a2-4874-93e6-151f1836ffb7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.202986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.203949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.210890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.211390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.212307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.215605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.324987 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3160/6391] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:26 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:26.326343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.340461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9d3ec-1d5d-4ba9-bb1c-ad7d9b9800cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.412617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:26.464824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.465585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.466761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.467059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.472287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.477387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.539229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.548508 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3232/6392] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:26 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:26.554183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dd763-d72c-42b4-942d-17f3430a89df admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.572215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.573512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.573800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.574085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.576010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.580643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.605421 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3161/6393] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:26.610237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.623998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e55323-4650-49fe-a234-525ea98da1af admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.631154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.631777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.632053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.632485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.633373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.635648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3976ef-9d46-4462-8c85-4c0580739c05 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.652903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3233/6394] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:13:26.654255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:26.674028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb1e2a-3142-4c34-8ad2-03c1664b99a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:26.688092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:26.689068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.689459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.689868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] PATH_INFO: `/v3/projects/d9909115070a42bda974f75d71fac5f7/users/6f82a4ffd97f4d349ba785d063615ebb/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.727535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9909115070a42bda974f75d71fac5f7, user_id=6f82a4ffd97f4d349ba785d063615ebb, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:26.740897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:26.771762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3162/6395] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:26 2022] PUT /identity/v3/projects/d9909115070a42bda974f75d71fac5f7/users/6f82a4ffd97f4d349ba785d063615ebb/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:26.784430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:26.784978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:26.784978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:26.868540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] MFA Rules not processed for user `6f82a4ffd97f4d349ba785d063615ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:26.870286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:27.058441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3234/6396] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:27.063687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.173885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ceb48-5c89-4105-a847-eb10d6c55f93 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.179246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.179899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.180235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.180512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.181834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.185805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.242073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3163/6397] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:27 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:27.246286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.262550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e74d9e-b6fd-4b5f-9860-5b27d70c6a2e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.360871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.361410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.361809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.362089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.362769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.367398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.398580 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3235/6398] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:27 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:27.402912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.423289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db778-a4ec-4e3a-ad63-eb784570f9f2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.447941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.448563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.449041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.449326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.450752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.452500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.457373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:27.471883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3164/6399] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:27.475687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.509160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdeacb8-207a-4fdc-aa91-8f0387043888 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.523796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.524583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.524990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.526082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.526973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.529159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffa88a-96c8-4110-9713-177cbdbcfce2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.562644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.585348 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3236/6400] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:27.586686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b165-6c82-4d1c-bf30-ad7602b6a976 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.600284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.601099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.604134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.605562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] PATH_INFO: `/v3/projects/f35b09d86e374d618b2ae64bd447bfea/users/74fa6bb3de124059a895b4227a9ee77c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.641626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35b09d86e374d618b2ae64bd447bfea, user_id=74fa6bb3de124059a895b4227a9ee77c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.644645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.665885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3165/6401] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:27 2022] PUT /identity/v3/projects/f35b09d86e374d618b2ae64bd447bfea/users/74fa6bb3de124059a895b4227a9ee77c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:27.681232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.681596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.681810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.723002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] MFA Rules not processed for user `74fa6bb3de124059a895b4227a9ee77c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:27.724937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:27.751224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:27.807903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3237/6402] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:27.813392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:27.922139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e995a4-3a6e-4442-905d-3b8585698ab4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:27.938637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:27.939649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:27.940181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:27.940635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:27.941459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:27.943453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:27.987585 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3166/6403] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:27 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:27.991613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.011738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739a229-c06a-45ed-9dd8-4d9fd07f48f7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:28.130129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:28.130993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.132921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.133404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.134787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:28.142391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:28.211518 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3238/6404] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:27 2022] POST /identity/v3/users => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:28.216339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.235834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d251ee73-80ae-4d06-8b14-91ffce0516d0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:28.242821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:28.243555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.243983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.244378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.245127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:28.247025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:28.271641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3167/6405] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:28.275619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.296639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0d6b5-2030-4e39-9eb2-b2dbda956161 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:28.302230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:28.303428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.303855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.304581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] PATH_INFO: `/v3/projects/b811c10f8168460c8d981462b2ae42e7/users/1096db5b9694448391a5042d540448b6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.332404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b811c10f8168460c8d981462b2ae42e7, user_id=1096db5b9694448391a5042d540448b6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:28.352013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:28.382747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3239/6406] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:28 2022] PUT /identity/v3/projects/b811c10f8168460c8d981462b2ae42e7/users/1096db5b9694448391a5042d540448b6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:28.386764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.399845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9119c59-3520-4a9c-86e2-f6ff46b5c5ff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:28.536307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:28.538227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.539734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.540033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.542120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:28.547172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b92002-34c8-4beb-999a-7ebabd237e84 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:28.574162 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3168/6407] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:28.575994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.592699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21f0fd-75fd-463f-87d2-6c3c6b7254d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:28.604455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:28.605598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.606105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.606891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] PATH_INFO: `/v3/projects/b811c10f8168460c8d981462b2ae42e7/users/1096db5b9694448391a5042d540448b6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.614323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b811c10f8168460c8d981462b2ae42e7, user_id=1096db5b9694448391a5042d540448b6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:28.617854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb72e2f-99bc-455d-9ff5-53aff739824b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:28.646540 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3240/6408] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:28 2022] PUT /identity/v3/projects/b811c10f8168460c8d981462b2ae42e7/users/1096db5b9694448391a5042d540448b6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:28.653553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.654549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.656565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.712939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] MFA Rules not processed for user `1096db5b9694448391a5042d540448b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:28.714565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:28.779150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:28.779438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:28.779715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:28.819147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:28.847025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] MFA Rules not processed for user `372fe818c768428aac3d9e88432ddd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:28.848255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:28.888861 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3169/6409] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:28.890725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:28.911436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:28.989012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:29.002608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3241/6410] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:29.009063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:29.010830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:29.029157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f712f-d5cb-4b08-b1d9-987ff8d74277 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:29.033504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:29.034486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:29.035027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:29.035548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:29.143639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012d0a8-8c2a-4ab0-b58c-9c19dd166877 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:29.154355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:29.156129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:29.156807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:29.157085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:29.157355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:29.160221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:29.165904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:29.180918 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3170/6411] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:29.323998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:29.338983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d302c96-4509-4cfc-b64a-094f89f8e854 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:29.378216 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3242/6412] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:30.529264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:30.531537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:30.647437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea901350-6734-4e2d-9f3f-4cf1d446e67a service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:30.654662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:30.655903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:30.656266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:30.656670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] PATH_INFO: `/v3/users/645cbc3ddcde476d89433ff04b5feb5e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:30.658190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645cbc3ddcde476d89433ff04b5feb5e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:30.660827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:30.731506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:30.791585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645cbc3ddcde476d89433ff04b5feb5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:30.792333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645cbc3ddcde476d89433ff04b5feb5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:30.844732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645cbc3ddcde476d89433ff04b5feb5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:30.867179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645cbc3ddcde476d89433ff04b5feb5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:30.871576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645cbc3ddcde476d89433ff04b5feb5e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:30.911444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3171/6413] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:30 2022] DELETE /identity/v3/users/645cbc3ddcde476d89433ff04b5feb5e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:30.916038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:30.916558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:30.916558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:30.962959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:30.965539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:31.123628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3243/6414] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:31.152760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:31.154541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:31.338231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2032c7-71c3-4da3-a43b-0c978a1e7765 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:31.358124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:31.358823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:31.359108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:31.359386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:31.384639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:31.387683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b0820-72f9-4491-b930-373027057d7d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:31.423704 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3172/6415] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:32.051054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:32.053524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:32.073036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7853b-0ac5-4871-96de-46090822871c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:32.081603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:32.084245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.085440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.085925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] PATH_INFO: `/v3/projects/ba9007e00fac49c7a50b5fd15572d276` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.088386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9007e00fac49c7a50b5fd15572d276)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:32.091283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:32.209339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9007e00fac49c7a50b5fd15572d276', 'request_id': 'req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:32.211278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:32.226076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9007e00fac49c7a50b5fd15572d276', 'request_id': 'req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:32.237733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9007e00fac49c7a50b5fd15572d276', 'request_id': 'req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:32.266367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] The token cache is being invalidate because project ba9007e00fac49c7a50b5fd15572d276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:32.266850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:32.269685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-25a6c0f7-2dbc-4b9a-a778-5a2cb2ebc0aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:32.272456 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3244/6416] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:32 2022] DELETE /identity/v3/projects/ba9007e00fac49c7a50b5fd15572d276 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:32.279503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.281050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.281657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.309374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:32.319263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:32.444804 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3173/6417] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:32.449161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.450186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.450640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.509889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:32.511463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:32.542770 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3245/6418] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:32.542980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:32.556107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379404-5bb3-4237-a4ca-c514ba8a3e10 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:32.566123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:32.566995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.567282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.567559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.568643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:32.568994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:32.571225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:32.647524 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3174/6419] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:32.648967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:32.665395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd22c3-c45c-4c92-80e4-c049daf22176 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:32.681253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:32.681880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.682167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.682446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.683109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:32.691087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:32.767901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3246/6420] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:32 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:32.770260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:32.805461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c449f-ab45-4565-adae-b6380ae3295e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:32.894526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:32.966621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:32.967853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:32.968586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:32.969620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:32.970605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:32.976062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.039627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3175/6421] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:32 2022] POST /identity/v3/users => generated 295 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:33.043876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.065527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863d091-17dc-49ec-9d25-2dd49803823a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:33.074173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:33.076175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.077461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.078170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.079085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:33.082759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.107371 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3247/6422] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:33.112139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.125631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de8d85-4f56-4d04-baa4-64ac8e8827aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:33.133772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:33.133772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.133772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.133772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.135279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:33.136901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525acd7-cc6c-4f59-8a90-7f8f73651bd6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.156995 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3176/6423] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:33.160683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.179479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056cee-2787-41ed-a4ac-ed5e80bd2ae2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:33.187989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:33.189552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.189846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.190174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] PATH_INFO: `/v3/projects/15c55e156af448b184f8b184650da5f3/users/0f254bac698e47ed8ecf3a403da67d9b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.212217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c55e156af448b184f8b184650da5f3, user_id=0f254bac698e47ed8ecf3a403da67d9b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:33.215478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.247314 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3248/6424] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:33 2022] PUT /identity/v3/projects/15c55e156af448b184f8b184650da5f3/users/0f254bac698e47ed8ecf3a403da67d9b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:33.261898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.262089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.262300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.323116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.323116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:33.339879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] MFA Rules not processed for user `0f254bac698e47ed8ecf3a403da67d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:33.341131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:33.427430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7be2d-9d84-44b9-8734-1b0d19290227 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:33.433297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:33.434037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.434352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.434746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] PATH_INFO: `/v3/users/6f82a4ffd97f4d349ba785d063615ebb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.436050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f82a4ffd97f4d349ba785d063615ebb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:33.439207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.467046 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3177/6425] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:33.475631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.477872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:33.525562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f82a4ffd97f4d349ba785d063615ebb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:33.526253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f82a4ffd97f4d349ba785d063615ebb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:33.539966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f82a4ffd97f4d349ba785d063615ebb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:33.561210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d43e2-3559-45aa-8d54-a3d826cec9f9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:33.568408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:33.568874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f82a4ffd97f4d349ba785d063615ebb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:33.569630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.570246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.570750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.571155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f82a4ffd97f4d349ba785d063615ebb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:33.575234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:33.588362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:33.591260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:33.593678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e3d99-0080-4e88-beba-b007b1b37f77 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:33.605600 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3178/6426] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:13:33.610595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3249/6427] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:33 2022] DELETE /identity/v3/users/6f82a4ffd97f4d349ba785d063615ebb => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:33.615319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:33.615918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:33.616483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:33.651407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:33.652670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:33.715006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:33.746401 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3179/6428] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:33.779367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:33.783349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:34.144627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05dc40-09c4-421d-b8de-f44771fe1e2e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:34.148509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:34.149239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:34.149709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:34.150210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:34.162703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:34.166273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:34.176457 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3250/6429] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:35.231548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:35.234013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:35.252571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcca29c-e3d1-488b-8d4a-bd059980b0d1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:35.263061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:35.264208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:35.266001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:35.269263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] PATH_INFO: `/v3/projects/d9909115070a42bda974f75d71fac5f7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:35.276818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9909115070a42bda974f75d71fac5f7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:35.282768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:35.378561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9909115070a42bda974f75d71fac5f7', 'request_id': 'req-9f8b8555-036c-4be0-be99-454f04cbca87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.387057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9909115070a42bda974f75d71fac5f7', 'request_id': 'req-9f8b8555-036c-4be0-be99-454f04cbca87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.399665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9909115070a42bda974f75d71fac5f7', 'request_id': 'req-9f8b8555-036c-4be0-be99-454f04cbca87'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.416659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] The token cache is being invalidate because project d9909115070a42bda974f75d71fac5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:35.416963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.417632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.422945 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3180/6430] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:35 2022] DELETE /identity/v3/projects/d9909115070a42bda974f75d71fac5f7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:35.424147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:35.444856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8745e-80d7-41cb-83c5-6d08056af1bd service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:35.534628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:35.535442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:35.535832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:35.536325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] PATH_INFO: `/v3/users/74fa6bb3de124059a895b4227a9ee77c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:35.537811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fa6bb3de124059a895b4227a9ee77c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:35.539950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:35.624316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fa6bb3de124059a895b4227a9ee77c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.625947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fa6bb3de124059a895b4227a9ee77c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.656748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fa6bb3de124059a895b4227a9ee77c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.677242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fa6bb3de124059a895b4227a9ee77c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.678048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fa6bb3de124059a895b4227a9ee77c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:35.700900 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3251/6431] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:35 2022] DELETE /identity/v3/users/74fa6bb3de124059a895b4227a9ee77c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:36.883426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:36.885118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:36.904794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b8555-036c-4be0-be99-454f04cbca87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:37.022149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:37.023162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:37.023662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:37.024140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] PATH_INFO: `/v3/projects/f35b09d86e374d618b2ae64bd447bfea` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:37.028367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35b09d86e374d618b2ae64bd447bfea)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:37.031635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:37.157480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35b09d86e374d618b2ae64bd447bfea', 'request_id': 'req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.187726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35b09d86e374d618b2ae64bd447bfea', 'request_id': 'req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.221081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35b09d86e374d618b2ae64bd447bfea', 'request_id': 'req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.234252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:37.243207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] The token cache is being invalidate because project f35b09d86e374d618b2ae64bd447bfea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:37.243843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.245014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.247263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3181/6432] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:36 2022] DELETE /identity/v3/projects/f35b09d86e374d618b2ae64bd447bfea => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:37.251214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:37.254152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:37.287245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e8315-1736-42eb-88f0-1d59793821a2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:37.307015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:37.385695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:37.394703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:37.394703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:37.394703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] PATH_INFO: `/v3/users/1096db5b9694448391a5042d540448b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:37.432872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1096db5b9694448391a5042d540448b6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:37.435015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:37.517132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1096db5b9694448391a5042d540448b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.517927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1096db5b9694448391a5042d540448b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.531643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1096db5b9694448391a5042d540448b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.551783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1096db5b9694448391a5042d540448b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.553699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-562ede82-56bf-4b90-81b5-dbfce8e0c075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1096db5b9694448391a5042d540448b6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:37.584226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3252/6433] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:37 2022] DELETE /identity/v3/users/1096db5b9694448391a5042d540448b6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:39.062855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:39.064631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:39.095675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6a2f-7c8d-4c0f-814c-925f0b502e31 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:39.301220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:39.302046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:39.302605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:39.302980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] PATH_INFO: `/v3/projects/b811c10f8168460c8d981462b2ae42e7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:39.305325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b811c10f8168460c8d981462b2ae42e7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:39.308125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:39.424757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b811c10f8168460c8d981462b2ae42e7', 'request_id': 'req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:39.436212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b811c10f8168460c8d981462b2ae42e7', 'request_id': 'req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:39.454269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b811c10f8168460c8d981462b2ae42e7', 'request_id': 'req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:39.496464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] The token cache is being invalidate because project b811c10f8168460c8d981462b2ae42e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:39.496760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:39.498346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f518c8a8-7dde-45e9-b9a1-b12ee64467b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:39.503235 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3182/6434] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:39 2022] DELETE /identity/v3/projects/b811c10f8168460c8d981462b2ae42e7 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:39.508726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:39.509277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:39.509277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:39.547037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:39.548287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:39.741114 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3253/6435] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:39.748448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:39.749459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:39.749927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:39.805931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:39.807540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:39.827384 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3183/6436] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:39.828834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:39.854445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08f8e6-4299-49db-8b59-ba23925cee78 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:39.871445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:39.872239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:39.872372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:39.873216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:39.873354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:39.875085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:39.876928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:39.912727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:39.925415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3254/6437] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:39.932779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d7541-89ec-4a82-b431-66557ba78e22 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:39.939543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:39.940122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:39.940466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:39.940747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:39.941461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:39.944203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.021022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3184/6438] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:39 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:40.025641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:40.042629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70539cb0-a991-44d6-b44f-c02d5d83a579 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:40.137858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:40.227340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:40.227977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.228259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.228531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.229169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:40.231144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.306102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3255/6439] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:40 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:40.310673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:40.344433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8aab8f-5a39-49b5-bfb7-a3a36cb5fa1c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:40.352747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:40.354145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.354885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.356531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.356531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:40.358934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.396001 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3185/6440] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:40.401888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:40.420041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e14f0-b1a6-4498-a54b-24a7ac389cb7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:40.439133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:40.439925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.440301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.440678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.441505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:40.443578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239875f-f0b6-49d9-8f3d-0f41d37f5f83 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.462347 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3256/6441] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:40.467140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:40.492080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef961334-fcc2-4ce5-ac2a-b468874d1649 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:40.500853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:40.502159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.502716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.503233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] PATH_INFO: `/v3/projects/63c554bf345d4d36839a42fd7c38a201/users/b41b7e2fd2ad44d6bf5e79a9a35cd60f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.555608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c554bf345d4d36839a42fd7c38a201, user_id=b41b7e2fd2ad44d6bf5e79a9a35cd60f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:40.565017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.611498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3186/6442] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:40 2022] PUT /identity/v3/projects/63c554bf345d4d36839a42fd7c38a201/users/b41b7e2fd2ad44d6bf5e79a9a35cd60f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:40.632006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.632585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.632585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.687282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] MFA Rules not processed for user `b41b7e2fd2ad44d6bf5e79a9a35cd60f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:40.691108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:40.819377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3257/6443] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:40.833077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:40.836600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:40.883228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:40.939896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c92c07-45c6-4233-9541-f27f8fd8e933 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:40.944487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:40.945585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:40.946099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:40.946822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:40.961819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:40.963769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3c11e-e7aa-498f-bf86-c5b3b82e96d7 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:40.992767 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3187/6444] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:41.867379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:41.868896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:41.971871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a6091-6ef9-4da5-8461-66f56392fa1a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:41.978348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:41.978578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:41.978784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:41.979062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] PATH_INFO: `/v3/users/8e03d4aced164fd1be95a02b07655809` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:41.980858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e03d4aced164fd1be95a02b07655809)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:41.984142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:42.038684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:42.081405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e03d4aced164fd1be95a02b07655809'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:42.082110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e03d4aced164fd1be95a02b07655809'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:42.095590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e03d4aced164fd1be95a02b07655809'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:42.121039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e03d4aced164fd1be95a02b07655809'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:42.123405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e03d4aced164fd1be95a02b07655809'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:42.148358 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3258/6445] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:41 2022] DELETE /identity/v3/users/8e03d4aced164fd1be95a02b07655809 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:42.152723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:42.153850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:42.154365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:42.179843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:42.181427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:42.306528 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3188/6446] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:42.372810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:42.375237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:42.455595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815fa534-162c-47fc-98a8-3fe969e3e777 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:42.460183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:42.460869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:42.461258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:42.461915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:42.471945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:42.473532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e17ee-1189-4137-bb75-ffd2969c8698 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:42.488176 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3259/6447] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:13:43.151732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.153470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:43.174485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f433d8-5d29-4750-a255-67273407bcd2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.184019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.184668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.184985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.191041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] PATH_INFO: `/v3/projects/bdf9dbda5d6447dda073dcc30c045403` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.192693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf9dbda5d6447dda073dcc30c045403)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.194452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.287200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf9dbda5d6447dda073dcc30c045403', 'request_id': 'req-ebb95aab-9b42-4573-8c80-15688d18a757'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:43.297113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf9dbda5d6447dda073dcc30c045403', 'request_id': 'req-ebb95aab-9b42-4573-8c80-15688d18a757'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:43.315262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf9dbda5d6447dda073dcc30c045403', 'request_id': 'req-ebb95aab-9b42-4573-8c80-15688d18a757'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:43.369552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] The token cache is being invalidate because project bdf9dbda5d6447dda073dcc30c045403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:43.370120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:43.375384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebb95aab-9b42-4573-8c80-15688d18a757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:43.377748 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3189/6448] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:43 2022] DELETE /identity/v3/projects/bdf9dbda5d6447dda073dcc30c045403 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:43.383746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.384212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.384212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.425031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:43.429155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:43.532240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3260/6449] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:43.539892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.540279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.540556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.563664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:43.565188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:43.574772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3190/6450] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:43.580606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.591237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d688fe-2d13-44d3-b7ce-bc1fab75df86 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.596762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.597369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.597811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.598082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.598757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:43.599058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.600674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.613323 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3261/6451] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:43.617964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.634680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4fe9c-6060-41bd-9f42-72df3890e27d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.636089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.636344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.636637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.637100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.637626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.640446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.673402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3191/6452] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:43 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:13:43.674559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.687464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99c0e2-df51-4ded-8d4e-2f961f93685c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.741145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:43.770924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.771299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.771579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.771922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.772550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.774735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.809551 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3262/6453] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:43 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:43.810059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.824248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd94d12-2e8b-432e-8ac6-3f019a9daf40 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.832705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.832705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.832705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.832705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.833325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.835711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.849746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3192/6454] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:43.853616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.877777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633a68b-218b-4198-8ba4-202246d9f6e7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.891206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.891921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.892201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.894902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.896025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.898724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bfe64-c1e0-40a8-acc7-16acda28df06 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:43.915665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3263/6455] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:43.919800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:43.934891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96450010-41d8-4c19-a822-ec20b5d817ac admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:43.942905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:43.943942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:43.944447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:43.944930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] PATH_INFO: `/v3/projects/b37d55810dcd46a69b760a658444b3b7/users/efd14950096140a091067e200301ae3e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:43.956272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:43.968247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37d55810dcd46a69b760a658444b3b7, user_id=efd14950096140a091067e200301ae3e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:43.973706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:44.002579 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3193/6456] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:43 2022] PUT /identity/v3/projects/b37d55810dcd46a69b760a658444b3b7/users/efd14950096140a091067e200301ae3e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:44.004704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:44.004991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:44.005268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:44.037284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] MFA Rules not processed for user `efd14950096140a091067e200301ae3e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:44.038482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:44.132007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3264/6457] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:44.152251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:44.152251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:44.230205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b147df-2642-46e5-aa4d-398d9b1dad01 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:44.235287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:44.235966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:44.236314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:44.236660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:44.246688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:44.249286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b845e-8a55-4f46-bdf7-b949b35018d1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:44.259317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3194/6458] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:44.921030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:44.922531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:45.016552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7a70-9db2-4d5a-87b5-dce2434476b4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:45.023773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:45.024965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:45.025610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:45.026190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] PATH_INFO: `/v3/users/b41b7e2fd2ad44d6bf5e79a9a35cd60f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:45.027580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41b7e2fd2ad44d6bf5e79a9a35cd60f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:45.030430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:45.094433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41b7e2fd2ad44d6bf5e79a9a35cd60f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:45.095520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41b7e2fd2ad44d6bf5e79a9a35cd60f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:45.099026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:45.105684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41b7e2fd2ad44d6bf5e79a9a35cd60f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:45.124978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41b7e2fd2ad44d6bf5e79a9a35cd60f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:45.126605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41b7e2fd2ad44d6bf5e79a9a35cd60f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:45.144693 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3265/6459] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:44 2022] DELETE /identity/v3/users/b41b7e2fd2ad44d6bf5e79a9a35cd60f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:45.146518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:45.146948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:45.147233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:45.170899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:45.173101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:45.244896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3195/6460] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:45.281274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:45.283706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:45.364038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdf92-900d-430f-91d0-c597a5d984b2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:45.368714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:45.369579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:45.370081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:45.370462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:45.383149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:45.384866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19057994-a72d-4ff7-87c1-bce33dfc267c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:45.399357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3266/6461] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:13:46.934530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:46.936044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:46.948986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652a2f-3272-4d07-ba07-101c0f19e451 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:46.954591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:46.955228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:46.955504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:46.955952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] PATH_INFO: `/v3/projects/63c554bf345d4d36839a42fd7c38a201` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:46.957287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c554bf345d4d36839a42fd7c38a201)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:46.959457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.037279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c554bf345d4d36839a42fd7c38a201', 'request_id': 'req-14ad67ef-5b6e-40fa-870a-1740ac125ca4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:47.043256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c554bf345d4d36839a42fd7c38a201', 'request_id': 'req-14ad67ef-5b6e-40fa-870a-1740ac125ca4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:47.055176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c554bf345d4d36839a42fd7c38a201', 'request_id': 'req-14ad67ef-5b6e-40fa-870a-1740ac125ca4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:47.070055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] The token cache is being invalidate because project 63c554bf345d4d36839a42fd7c38a201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:47.070343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:47.071355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14ad67ef-5b6e-40fa-870a-1740ac125ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:47.073078 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3196/6462] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:46 2022] DELETE /identity/v3/projects/63c554bf345d4d36839a42fd7c38a201 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:47.080632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.081183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.081295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.128442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:47.130309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:47.236091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3267/6463] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:47.244913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.245207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.245498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.267477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:47.269077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:47.280235 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3197/6464] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:47.284641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.295875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20283232-d5a5-4d35-b968-f1f2a65d4baf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.301975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.302796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.303073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.303346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.304645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:47.304645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.306986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.321644 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3268/6465] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:47.325899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.336784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dd27e-e00f-481b-964f-6e3cc85f0304 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.341901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.343451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.343805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.344151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.344902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.346972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.352590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:47.395791 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3198/6466] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:47 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:47.397338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.410827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e39756-d3a1-4903-9c58-5324eaead3a8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.472522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:47.479003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.480116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.482161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.483394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.483930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.486169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.515124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3269/6467] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:47 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:47.519222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.533279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfcbf-0581-4381-86e7-7c51a27221ff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.538340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.539386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.539893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.540377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.541221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.543431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.556213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3199/6468] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:13:47.559953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.573089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af893a5-ab05-41b1-b600-03246b31504d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.579140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.579398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.579819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.580256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.580919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.582582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90d5bd-c4ea-4039-b682-c84f7403e7fd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.599044 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3270/6469] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:47.599990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.612306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24010780-b083-42ed-85d3-4d055f96fd3e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.618602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.619650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.620161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.620814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] PATH_INFO: `/v3/projects/bd2437f4fc2d42babc3ad8758e67c3f4/users/9acaeb0f607f40d6b35f9bc72a356f95/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.656621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2437f4fc2d42babc3ad8758e67c3f4, user_id=9acaeb0f607f40d6b35f9bc72a356f95, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.659764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.688253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3200/6470] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:47 2022] PUT /identity/v3/projects/bd2437f4fc2d42babc3ad8758e67c3f4/users/9acaeb0f607f40d6b35f9bc72a356f95/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:47.688918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.689206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.689479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.758747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] MFA Rules not processed for user `9acaeb0f607f40d6b35f9bc72a356f95`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:47.759992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:47.873366 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3271/6471] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:47.887847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:47.891026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:47.969642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2d27-8366-49d8-b73a-574d5e2f4ba1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:47.975071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:47.975761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:47.976129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:47.977416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:47.987217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:47.988688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4aa4e-e46a-48ba-97f9-fc2b1f82eeef service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:47.999075 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3201/6472] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:49.821721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:49.934593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208bb97-4120-4170-a89b-b9b0e77af79d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:49.943436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:49.944671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:49.944671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:49.944996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] PATH_INFO: `/v3/users/9acaeb0f607f40d6b35f9bc72a356f95` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:49.946295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acaeb0f607f40d6b35f9bc72a356f95)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:49.948978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:50.037729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acaeb0f607f40d6b35f9bc72a356f95'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:50.044279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acaeb0f607f40d6b35f9bc72a356f95'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:50.065387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acaeb0f607f40d6b35f9bc72a356f95'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:50.096348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:50.113027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acaeb0f607f40d6b35f9bc72a356f95'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:50.113901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acaeb0f607f40d6b35f9bc72a356f95'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:50.141689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3272/6473] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:49 2022] DELETE /identity/v3/users/9acaeb0f607f40d6b35f9bc72a356f95 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:50.156932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:50.157822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:50.158300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:50.213225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:50.215098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:50.351386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3202/6474] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:50.392495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:50.394267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:50.543662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a76e9d-9aa7-48db-9930-a6f4c76e82e4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:50.549539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:50.550727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:50.551198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:50.551778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:50.567632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:50.577291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40924b3c-a8f8-4038-bccc-73959c2f767b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:50.597041 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3273/6475] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:51.752393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:51.753893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:51.781585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad640671-e1ce-45a0-b5ea-e06f1545c737 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:51.786706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:51.787360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:51.787644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:51.787920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] PATH_INFO: `/v3/projects/bd2437f4fc2d42babc3ad8758e67c3f4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:51.790471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2437f4fc2d42babc3ad8758e67c3f4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:51.792680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:51.928676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:51.931593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2437f4fc2d42babc3ad8758e67c3f4', 'request_id': 'req-54ae1c20-d935-4485-8deb-6ce93daab546'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:51.944268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2437f4fc2d42babc3ad8758e67c3f4', 'request_id': 'req-54ae1c20-d935-4485-8deb-6ce93daab546'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:51.978650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2437f4fc2d42babc3ad8758e67c3f4', 'request_id': 'req-54ae1c20-d935-4485-8deb-6ce93daab546'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:51.995944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] The token cache is being invalidate because project bd2437f4fc2d42babc3ad8758e67c3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:51.996650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:51.997555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-54ae1c20-d935-4485-8deb-6ce93daab546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.000477 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3203/6476] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:51 2022] DELETE /identity/v3/projects/bd2437f4fc2d42babc3ad8758e67c3f4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:52.006311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.006903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.006903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.040791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:52.043310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:52.226266 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3274/6477] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:52.237519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.237995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.238990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.270138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.275294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:52.303111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:52.306746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda86e0a-6ae6-4769-879b-c6c52d4f76cb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:52.307168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:52.314034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:52.317010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.317495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.318281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] PATH_INFO: `/v3/users/f9bee7e327e2490eaa445ec2f61a617b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.322051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bee7e327e2490eaa445ec2f61a617b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:52.325563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:52.329530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3204/6478] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:52.334569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.353626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20e29e-3962-4dfc-b4ef-3741a7323397 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:52.372769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:52.373450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.373736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.374064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.374852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:52.375161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:52.380325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:52.415960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3205/6479] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:52.425782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bee7e327e2490eaa445ec2f61a617b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.426524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bee7e327e2490eaa445ec2f61a617b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.435807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.437977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:52.443173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bee7e327e2490eaa445ec2f61a617b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.459801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbe645-3e13-4243-b849-ed680e3fbe64 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:52.466311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:52.468006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.468147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bee7e327e2490eaa445ec2f61a617b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.468681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.469276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.469704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bee7e327e2490eaa445ec2f61a617b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:52.471883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:52.474260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:52.482748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:52.501436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3275/6480] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:52 2022] DELETE /identity/v3/users/f9bee7e327e2490eaa445ec2f61a617b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:13:52.556805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.557065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3206/6481] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:52 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:52.733328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36994f10-5b39-4ddb-ba99-7a309f22ee0f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:52.825604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:52.826745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.828381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.828764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.830214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:52.834011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:52.890039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3276/6482] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:52 2022] POST /identity/v3/users => generated 292 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:13:52.893905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.918563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c129424-4c86-4353-8032-91eb8074a032 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:52.941734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:52.943551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:52.947820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:52.948491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:52.949454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:52.954586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:52.979862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3207/6483] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:52.980231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:52.996573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fef03-f78c-4c5d-adbf-eabe26483492 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:53.007873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:53.009503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.009503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.009503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.010068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:53.011991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:53.041059 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3277/6484] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:53.059419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:53.109803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511892e-adff-40c0-8778-3663c44ab03e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:53.132125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:53.135030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.136126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.138746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] PATH_INFO: `/v3/projects/ef49600347f949c7b2b58813046e1fe2/users/a748f800fd74471280063fd257cc0679/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.179945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef49600347f949c7b2b58813046e1fe2, user_id=a748f800fd74471280063fd257cc0679, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:53.183619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae88c-59dd-4ee1-be75-077895e69217 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:53.204660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:53.207298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:53.255472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e62f4-2cf2-4300-b2e4-a2e3c9bb0e7b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:53.256100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3208/6485] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:53 2022] PUT /identity/v3/projects/ef49600347f949c7b2b58813046e1fe2/users/a748f800fd74471280063fd257cc0679/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:53.278906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c124e-d426-4004-bd77-393569566fbe None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.280527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c124e-d426-4004-bd77-393569566fbe None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.281450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c124e-d426-4004-bd77-393569566fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.376522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-229c124e-d426-4004-bd77-393569566fbe None None] MFA Rules not processed for user `a748f800fd74471280063fd257cc0679`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:53.378480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-229c124e-d426-4004-bd77-393569566fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:53.472897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:53.475817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.476103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.476660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] PATH_INFO: `/v3/projects/2e7563fe5cfe4b5ba1783d9ec26d5009` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.481506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7563fe5cfe4b5ba1783d9ec26d5009)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:53.488500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:53.562891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:53.593103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3209/6486] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:53.601511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c124e-d426-4004-bd77-393569566fbe None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:53.603352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-229c124e-d426-4004-bd77-393569566fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:53.609945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7563fe5cfe4b5ba1783d9ec26d5009', 'request_id': 'req-05a39d86-16ee-4864-8d4d-baa36635c623'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:53.626022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7563fe5cfe4b5ba1783d9ec26d5009', 'request_id': 'req-05a39d86-16ee-4864-8d4d-baa36635c623'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:53.635400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7563fe5cfe4b5ba1783d9ec26d5009', 'request_id': 'req-05a39d86-16ee-4864-8d4d-baa36635c623'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:53.657053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] The token cache is being invalidate because project 2e7563fe5cfe4b5ba1783d9ec26d5009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:53.657405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:53.659412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:53.661936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3278/6487] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:53 2022] DELETE /identity/v3/projects/2e7563fe5cfe4b5ba1783d9ec26d5009 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:53.666991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:53.668978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:53.712611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:53.734844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c124e-d426-4004-bd77-393569566fbe None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:53.860968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a39d86-16ee-4864-8d4d-baa36635c623 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:53.880487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:53.886044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.886427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.886797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] PATH_INFO: `/v3/users/372fe818c768428aac3d9e88432ddd5a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.895876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372fe818c768428aac3d9e88432ddd5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:53.898125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:53.902460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:53.903124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:53.903805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:53.905032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:53.907303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:54.017862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372fe818c768428aac3d9e88432ddd5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:54.018468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372fe818c768428aac3d9e88432ddd5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:54.035247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372fe818c768428aac3d9e88432ddd5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:54.042538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:54.045463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:54.058013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372fe818c768428aac3d9e88432ddd5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:54.058921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372fe818c768428aac3d9e88432ddd5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:54.076526 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3210/6488] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:54.088307 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3279/6489] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:53 2022] DELETE /identity/v3/users/372fe818c768428aac3d9e88432ddd5a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:55.477063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:55.494849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e97a-0cde-43d8-a1fe-a864f3dfa23a service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:55.704010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:55.704808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:55.705609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:55.705892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] PATH_INFO: `/v3/projects/e64ad41b1eb54003bfef94546de43af2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:55.707800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e64ad41b1eb54003bfef94546de43af2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:55.714164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:55.807950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e64ad41b1eb54003bfef94546de43af2', 'request_id': 'req-c0303828-7268-4ce5-aa1f-2fe0070bea2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:55.816531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e64ad41b1eb54003bfef94546de43af2', 'request_id': 'req-c0303828-7268-4ce5-aa1f-2fe0070bea2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:55.819928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:55.826666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e64ad41b1eb54003bfef94546de43af2', 'request_id': 'req-c0303828-7268-4ce5-aa1f-2fe0070bea2a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:55.853568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] The token cache is being invalidate because project e64ad41b1eb54003bfef94546de43af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:55.854104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:55.854920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0303828-7268-4ce5-aa1f-2fe0070bea2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:55.866565 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3211/6490] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:55 2022] DELETE /identity/v3/projects/e64ad41b1eb54003bfef94546de43af2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:56.459972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:56.461672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:56.479264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0b746-5307-4b4b-9603-b90a9b299140 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:56.517821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:56.582528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:56.583455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:56.583902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:56.584339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] PATH_INFO: `/v3/users/a748f800fd74471280063fd257cc0679` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:56.588960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a748f800fd74471280063fd257cc0679)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:56.591182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:56.700346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a748f800fd74471280063fd257cc0679'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:56.701838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a748f800fd74471280063fd257cc0679'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:56.717510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a748f800fd74471280063fd257cc0679'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:56.758273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a748f800fd74471280063fd257cc0679'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:56.759168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a748f800fd74471280063fd257cc0679'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:56.791057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3280/6491] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:56 2022] DELETE /identity/v3/users/a748f800fd74471280063fd257cc0679 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:56.795995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:56.796613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:56.796916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:56.834906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:56.836475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:56.967488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3212/6492] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:57.007229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:57.009235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:57.195012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7602cc-4e01-44f8-97e2-4bbe5b167d16 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:57.200490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:57.204941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:57.211793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:57.212460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:57.264876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:57.266600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:57.296621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3281/6493] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:58.066428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.068626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.152230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.155418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.160466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84031f8-5441-4099-bc06-9b226a9b51d5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:58.171128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:58.172072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.172534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.172979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] PATH_INFO: `/v3/projects/ef49600347f949c7b2b58813046e1fe2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.174578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef49600347f949c7b2b58813046e1fe2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:58.176657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724de11-e22f-4666-a5f2-49685d7df3c5 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:58.178689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:58.191887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:58.192767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.193222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.193663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] PATH_INFO: `/v3/users/7a6fcbf3208747959c57681abb011b05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.196844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6fcbf3208747959c57681abb011b05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:58.200418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:58.297020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6fcbf3208747959c57681abb011b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.298119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6fcbf3208747959c57681abb011b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.304183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:58.307015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef49600347f949c7b2b58813046e1fe2', 'request_id': 'req-b5962cfb-b8c4-4b22-98b6-e00a239b154a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.311560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6fcbf3208747959c57681abb011b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.320333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef49600347f949c7b2b58813046e1fe2', 'request_id': 'req-b5962cfb-b8c4-4b22-98b6-e00a239b154a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.344249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef49600347f949c7b2b58813046e1fe2', 'request_id': 'req-b5962cfb-b8c4-4b22-98b6-e00a239b154a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.346590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6fcbf3208747959c57681abb011b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.348056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-85d25829-a779-4cf3-9e66-ab6358f7baa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6fcbf3208747959c57681abb011b05'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.368213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] The token cache is being invalidate because project ef49600347f949c7b2b58813046e1fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:13:58.368515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.376847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b5962cfb-b8c4-4b22-98b6-e00a239b154a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:13:58.378523 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3213/6494] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:58 2022] DELETE /identity/v3/projects/ef49600347f949c7b2b58813046e1fe2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:58.383799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033484f4-e789-4870-a8ca-6d8ad5ca67f2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.384290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033484f4-e789-4870-a8ca-6d8ad5ca67f2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.384760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033484f4-e789-4870-a8ca-6d8ad5ca67f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.408280 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3282/6495] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:13:58 2022] DELETE /identity/v3/users/7a6fcbf3208747959c57681abb011b05 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:58.414107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.415704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.415825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.443157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-033484f4-e789-4870-a8ca-6d8ad5ca67f2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:58.446601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-033484f4-e789-4870-a8ca-6d8ad5ca67f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.476330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:58.479526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.541451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:58.557902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3214/6496] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:58.566063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.566672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.567148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.630529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:58.635417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.662465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3215/6497] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:58.667643 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3283/6498] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:13:58.670455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.685787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.688972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:58.717062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bda7a8-c753-4e44-9857-39cdcace9b16 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:58.728152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:58.728983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.729327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.729690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.730559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:13:58.730945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:58.733249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:58.752939 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3216/6499] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:13:58.759887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.782734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d8eb9-587d-4800-b45e-ceaf26f72413 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:58.789217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:58.789834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.790114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.790387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.791051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:58.793029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:58.836359 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3217/6500] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:13:58 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:58.841479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:58.853714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1d453-ec8d-4cf9-a748-4868ceb4d1dc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:58.921657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:58.922782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:58.923195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:58.923478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:58.992312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:13:59.020652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef19136-7ccd-4a1d-b565-358f0ccf97e0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.028023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.029967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.030454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.031809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.032935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.036583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.043822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.046051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.062877 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3284/6501] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:59.071120 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3218/6502] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:13:58 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:13:59.075244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:59.089687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03f929-063d-4e0e-9e03-ab3b22d4d9f2 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.096072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.096242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.096476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.096681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.097279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.098959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.110372 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3285/6503] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:13:59.116345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:59.130536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91d3a8-9f91-4df4-ae0d-5c74c537fd01 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.136408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.137492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.137781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.138055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.139358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.141151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44818f42-b36c-40b6-888f-32cef675ea6f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.156156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3219/6504] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:13:59.159987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:59.172146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794004b-3e16-4ccb-98ff-e497b44e4cea admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.177532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.178423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.178587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.178795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] PATH_INFO: `/v3/projects/7fd14256923a4f19a4789b6f9553183f/users/57d8c826ee5e4f8898acb24b6843f9c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.198888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd14256923a4f19a4789b6f9553183f, user_id=57d8c826ee5e4f8898acb24b6843f9c0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.202521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.219279 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3286/6505] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:13:59 2022] PUT /identity/v3/projects/7fd14256923a4f19a4789b6f9553183f/users/57d8c826ee5e4f8898acb24b6843f9c0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:13:59.224100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.224517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.225652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.246242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:13:59.247502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:59.323498 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3220/6506] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:13:59.346289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:59.430332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703cc70-071c-402f-96f9-fd8b51e41bb4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.438781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.439801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.443073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.443465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.456795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.457977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.471665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3287/6507] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:13:59.886674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:13:59.889482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:13:59.903664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eebc6-158a-4a74-bf58-eb6f0c31faf9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:13:59.910027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:13:59.911002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:13:59.911353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:13:59.911697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] PATH_INFO: `/v3/projects/34187cd872e346cbb8c818b32fa72b34` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:13:59.914357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=34187cd872e346cbb8c818b32fa72b34)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:13:59.916548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:13:59.926558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:00.008676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34187cd872e346cbb8c818b32fa72b34', 'request_id': 'req-192714f6-b703-423d-ad02-a42e4c275efa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.021325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34187cd872e346cbb8c818b32fa72b34', 'request_id': 'req-192714f6-b703-423d-ad02-a42e4c275efa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.036659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34187cd872e346cbb8c818b32fa72b34', 'request_id': 'req-192714f6-b703-423d-ad02-a42e4c275efa'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.063594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] The token cache is being invalidate because project 34187cd872e346cbb8c818b32fa72b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:00.063953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.064628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.067546 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3221/6508] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:13:59 2022] DELETE /identity/v3/projects/34187cd872e346cbb8c818b32fa72b34 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:00.072136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:00.088979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f71df-4605-48eb-a50e-265339952c7b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:00.188830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:00.189475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:00.189840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:00.190357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] PATH_INFO: `/v3/users/b469649953a24842839d689b1f3885be` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:00.191766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b469649953a24842839d689b1f3885be)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:00.194226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:00.241624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:00.267568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b469649953a24842839d689b1f3885be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.268267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b469649953a24842839d689b1f3885be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.276528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b469649953a24842839d689b1f3885be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.293672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b469649953a24842839d689b1f3885be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.294571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fc916122-d08f-49d5-a984-49e2bf8047c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b469649953a24842839d689b1f3885be'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.314189 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3288/6509] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:00 2022] DELETE /identity/v3/users/b469649953a24842839d689b1f3885be => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:00.723915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:00.726426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:00.752079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192714f6-b703-423d-ad02-a42e4c275efa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:00.839772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:00.840128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:00.840980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:00.864377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:00.865869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:00.866477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:00.867036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] PATH_INFO: `/v3/projects/57580819362741b182ab7616cb8793ab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:00.869031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57580819362741b182ab7616cb8793ab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:00.871588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:00.876798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] MFA Rules not processed for user `57d8c826ee5e4f8898acb24b6843f9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:00.878177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:00.949262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57580819362741b182ab7616cb8793ab', 'request_id': 'req-a6923875-26fe-4308-91ac-2d3b471fcf83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.962724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57580819362741b182ab7616cb8793ab', 'request_id': 'req-a6923875-26fe-4308-91ac-2d3b471fcf83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.970718 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3289/6510] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:00.980356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57580819362741b182ab7616cb8793ab', 'request_id': 'req-a6923875-26fe-4308-91ac-2d3b471fcf83'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:00.980905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:00.983142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:01.002799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] The token cache is being invalidate because project 57580819362741b182ab7616cb8793ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:01.003222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:01.003836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6923875-26fe-4308-91ac-2d3b471fcf83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:01.007090 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3222/6511] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:00 2022] DELETE /identity/v3/projects/57580819362741b182ab7616cb8793ab => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:01.012082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fe6e7-0811-43ef-8c2c-d0d169a8f703 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.012370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fe6e7-0811-43ef-8c2c-d0d169a8f703 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.012648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fe6e7-0811-43ef-8c2c-d0d169a8f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.069682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:01.084885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a95fe6e7-0811-43ef-8c2c-d0d169a8f703 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:01.101120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a95fe6e7-0811-43ef-8c2c-d0d169a8f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:01.103674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4a40f-d2d7-4e5f-9beb-8c423511df12 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.204125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.206219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.206860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.207338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.210040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:01.223420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:01.247506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3223/6512] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:01.255991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.256351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.256541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.297084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:01.299103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:01.301371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.303772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.308364 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3224/6513] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:01.313215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.322233 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3290/6514] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:01.329455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236646-020a-4aed-b0ef-3e1b0e27cb93 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.336383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.337122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.337592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.338332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.339153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:01.339487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.341147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.360896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3225/6515] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:01.361466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.378317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63ac84-502b-4329-8f27-09e923319b71 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.385306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.386133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.386618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.387191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.388020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.390431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.430239 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3291/6516] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:01.431830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.444864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59d1e5-88b4-47dc-926a-77b9d888292e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.549032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.549729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.550051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.550425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.551103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.553198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.590333 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3226/6517] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:01 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:01.591874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.610028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df93e6-0f4c-4618-91d5-0fe3dcc69163 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.626955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.643890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.644074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.644184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.644866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.648266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.677936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3292/6518] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:01.681978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.701360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc363c-7527-4f58-a237-4b6b411855b0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.714631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.715327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.715656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.716054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.717591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.718883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c153eb-b709-4dc3-9c35-3692e9be11a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.737565 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3227/6519] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:01.741826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:01.759777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3611f6-13e0-4e23-b596-73e644e6ad36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:01.784225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:01.785026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.785318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.785595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] PATH_INFO: `/v3/projects/3329d177ef8e40d3b511b4656ba925a3/users/571351198eec43faa7a3b1abdde0c9e0/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.811650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3329d177ef8e40d3b511b4656ba925a3, user_id=571351198eec43faa7a3b1abdde0c9e0, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:01.814661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:01.839949 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3293/6520] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:01 2022] PUT /identity/v3/projects/3329d177ef8e40d3b511b4656ba925a3/users/571351198eec43faa7a3b1abdde0c9e0/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:01.851518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51979dab-0281-4356-a206-7a0e9df452da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:01.851893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51979dab-0281-4356-a206-7a0e9df452da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:01.852097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51979dab-0281-4356-a206-7a0e9df452da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:01.894238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-51979dab-0281-4356-a206-7a0e9df452da None None] MFA Rules not processed for user `571351198eec43faa7a3b1abdde0c9e0`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:01.895618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-51979dab-0281-4356-a206-7a0e9df452da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:01.959862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:01.976202 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3228/6521] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:02.012187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:02.015192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:02.103949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29209cca-0887-4985-a988-cc95c5be2472 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:02.109529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:02.111060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:02.111060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:02.111060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:02.123188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:02.126589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513508c0-8417-4997-b86d-adfc21505080 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:02.137317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3294/6522] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:04.800615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51979dab-0281-4356-a206-7a0e9df452da None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:04.801904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-51979dab-0281-4356-a206-7a0e9df452da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:04.902435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51979dab-0281-4356-a206-7a0e9df452da None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:04.912576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:04.913971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:04.914614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:04.915228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] PATH_INFO: `/v3/users/0f254bac698e47ed8ecf3a403da67d9b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:04.928335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f254bac698e47ed8ecf3a403da67d9b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:04.932010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:05.047108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f254bac698e47ed8ecf3a403da67d9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.047917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f254bac698e47ed8ecf3a403da67d9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.063689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f254bac698e47ed8ecf3a403da67d9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.078308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f254bac698e47ed8ecf3a403da67d9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.082103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f254bac698e47ed8ecf3a403da67d9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.105639 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3229/6523] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:04 2022] DELETE /identity/v3/users/0f254bac698e47ed8ecf3a403da67d9b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:05.111740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:05.112291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:05.112291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:05.136532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:05.138932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:05.149960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:05.151240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:05.227923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:05.272779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b1595-f4f1-4fe7-aa6b-a5869e741fa6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:05.286997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:05.288564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:05.289019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:05.289565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] PATH_INFO: `/v3/users/57d8c826ee5e4f8898acb24b6843f9c0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:05.291073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d8c826ee5e4f8898acb24b6843f9c0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:05.291238 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3295/6524] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:05.293619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:05.317708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:05.319999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:05.322363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:05.395211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d8c826ee5e4f8898acb24b6843f9c0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.400695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d8c826ee5e4f8898acb24b6843f9c0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.412972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d8c826ee5e4f8898acb24b6843f9c0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.423882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d8c826ee5e4f8898acb24b6843f9c0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.425285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d8c826ee5e4f8898acb24b6843f9c0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:05.464105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffcd0-e0c2-40dd-ac93-0532f61cc7ea None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:05.468764 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3230/6525] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:05 2022] DELETE /identity/v3/users/57d8c826ee5e4f8898acb24b6843f9c0 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:05.541848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:05.542487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:05.542817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:05.543092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:05.546295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:05.660430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:05.661963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:05.675915 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3296/6526] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:06.119028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.120836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.141443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f03049-54fb-4975-a38f-81b97828959a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.148402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.149901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.150440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.150917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] PATH_INFO: `/v3/projects/7fd14256923a4f19a4789b6f9553183f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.152789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd14256923a4f19a4789b6f9553183f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.161754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.199173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.201411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.220455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc3fd-12de-45f3-b4e1-1a5c9000f564 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.228155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.229160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.229652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.230078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] PATH_INFO: `/v3/projects/15c55e156af448b184f8b184650da5f3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.233471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c55e156af448b184f8b184650da5f3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.237333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.277640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd14256923a4f19a4789b6f9553183f', 'request_id': 'req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.295570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd14256923a4f19a4789b6f9553183f', 'request_id': 'req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.304069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd14256923a4f19a4789b6f9553183f', 'request_id': 'req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.318139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:06.328767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] The token cache is being invalidate because project 7fd14256923a4f19a4789b6f9553183f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:06.329063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.333680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c55e156af448b184f8b184650da5f3', 'request_id': 'req-c6043533-f568-4aff-940a-9b99694316bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.334830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ea0ad39-9b8d-4039-960d-8eb3dd4116d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.337689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3231/6527] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:06 2022] DELETE /identity/v3/projects/7fd14256923a4f19a4789b6f9553183f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:06.351423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbd8f9-21c6-49dc-8f0c-1a2505db67bb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.352086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbd8f9-21c6-49dc-8f0c-1a2505db67bb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.352584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbd8f9-21c6-49dc-8f0c-1a2505db67bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.355011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c55e156af448b184f8b184650da5f3', 'request_id': 'req-c6043533-f568-4aff-940a-9b99694316bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.364161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c55e156af448b184f8b184650da5f3', 'request_id': 'req-c6043533-f568-4aff-940a-9b99694316bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.393455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] The token cache is being invalidate because project 15c55e156af448b184f8b184650da5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:06.393958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.395139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c6043533-f568-4aff-940a-9b99694316bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:06.398859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3297/6528] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:06 2022] DELETE /identity/v3/projects/15c55e156af448b184f8b184650da5f3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:06.404178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151dac-278f-42cf-83ad-78c9196d7566 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.404556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151dac-278f-42cf-83ad-78c9196d7566 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.404893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151dac-278f-42cf-83ad-78c9196d7566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.451675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74fbd8f9-21c6-49dc-8f0c-1a2505db67bb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:06.453889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74fbd8f9-21c6-49dc-8f0c-1a2505db67bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.455110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59151dac-278f-42cf-83ad-78c9196d7566 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:06.459770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59151dac-278f-42cf-83ad-78c9196d7566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.571621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3298/6529] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:06.575107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.576321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.576819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.585232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:06.605762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3232/6530] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:06.613186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:06.614374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.615042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.615411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.615818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.627462 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3299/6531] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:06.633873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.649218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50247958-6ae1-4bfc-8115-aebb63276ec0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.652475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:06.654175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:06.661626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.662261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.662583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.662833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.663485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:06.663792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.665393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.677821 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3233/6532] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:06.678844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.691617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3300/6533] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:06.692758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.705356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9686666-d8e6-4fc8-b88d-ddb62b77fc83 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.721939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840a8f-e049-4797-a3c1-859c2274cfe6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.722534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.728185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.730921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.733400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.734111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:06.734412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.735753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.736084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.736393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.736655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.736925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.737657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.740228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.769762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3234/6534] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:06.774373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.790198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a29dd-f3f4-44d5-88a5-a49dc7cd0b75 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:06.801721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:06.802817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:06.804414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:06.804799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:06.817376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:06.818300 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3301/6535] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:06.822351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:06.827533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.899761 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3235/6536] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:06.900843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:06.963901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77f4c0-6138-4853-b680-3dad11f4f5ac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.002982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:07.054643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30492f0b-e34a-4ebf-be1a-c81d53d41939 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.065747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.066847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.067275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.067611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.068494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.074160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.074366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.075013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.075431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.075871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.076894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.081903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.131239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3236/6537] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:07.135474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.143193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3302/6538] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:06 2022] POST /identity/v3/users => generated 287 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:07.146462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.157317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031187d1-fb2d-4551-a95a-682471344da1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.170760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.172548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.173006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.173445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.174251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.175502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ff634-dd61-49a5-9ec8-bb98557fc994 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.176371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.188453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3237/6539] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:07.189379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.190091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.190478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.190880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.191781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.192775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.196027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.210447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b695d-f9d8-4663-a125-f2e7e40dda7a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.218932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.219826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.220276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.220705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.221360 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3303/6540] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:07.222268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.223585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.223962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.241999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4dae-b61e-44e4-803d-61a6d6a9d982 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.245294 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3238/6541] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:07.249754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.263844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.264720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.265170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.265632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.266453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.269684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.275110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788b26e-199d-49f9-b19e-6737268399a8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.282412 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3304/6542] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:07.286778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.295711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.296557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.296944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.297347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] PATH_INFO: `/v3/projects/c3d65fb083bf46fbaa1ac97228b5b4a2/users/2876ced64d6344e4a4be30a9b09f88d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.301770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0398-be83-4e43-a691-544a6dd3c0ed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.308208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.309114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.315154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.315724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] PATH_INFO: `/v3/projects/dfb55e300d9542d9a08c33731adce8af/users/9c1ebfb5f44948e38e01338ba19e34ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.330895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d65fb083bf46fbaa1ac97228b5b4a2, user_id=2876ced64d6344e4a4be30a9b09f88d1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.334280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816c1d2-58d1-4765-88f4-4cc91d2dce7a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.337065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb55e300d9542d9a08c33731adce8af, user_id=9c1ebfb5f44948e38e01338ba19e34ff, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.342561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43695d44-b90f-4bff-902c-7b20db965280 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.353967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3239/6543] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:07 2022] PUT /identity/v3/projects/c3d65fb083bf46fbaa1ac97228b5b4a2/users/2876ced64d6344e4a4be30a9b09f88d1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:07.360301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.360900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.361513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.375558 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3305/6544] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:07 2022] PUT /identity/v3/projects/dfb55e300d9542d9a08c33731adce8af/users/9c1ebfb5f44948e38e01338ba19e34ff/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:07.385808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.386124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.386641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.395291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:07.397407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:07.437655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] MFA Rules not processed for user `9c1ebfb5f44948e38e01338ba19e34ff`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:07.439827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:07.488119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3240/6545] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:07.509994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.511637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:07.517807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:07.533498 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3306/6546] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:07.563541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:07.565022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:07.603461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8519d-33fb-446f-a661-04eefc85f202 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.608669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.609927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.610421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.611057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.626748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.628737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a34860-01ea-4b8a-93ca-5e98e1b46061 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.642214 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3241/6547] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:07.672862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6d2d5-3054-4caf-9c56-e8482cf82129 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:07.678728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:07.678949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:07.679261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:07.679602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:07.691328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:07.692912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:07.705110 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3307/6548] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:08.677252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0de5-2c79-48ae-9e17-1f0d6b801a09 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:08.678586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0de5-2c79-48ae-9e17-1f0d6b801a09 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:08.679422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b0de5-2c79-48ae-9e17-1f0d6b801a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:08.706209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-985b0de5-2c79-48ae-9e17-1f0d6b801a09 None None] MFA Rules not processed for user `57c163c13444454c9a23703c08a9cc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:08.707915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-985b0de5-2c79-48ae-9e17-1f0d6b801a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:08.802684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3242/6549] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:08.845779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:08.856714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:08.865855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acad72-4b5a-40c4-b164-4730a484be47 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:08.871498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:08.872515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:08.873072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:08.873807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] PATH_INFO: `/v3/users/9c1ebfb5f44948e38e01338ba19e34ff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:08.875559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1ebfb5f44948e38e01338ba19e34ff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:08.877777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:08.976556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1ebfb5f44948e38e01338ba19e34ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:08.977317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1ebfb5f44948e38e01338ba19e34ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:08.989359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1ebfb5f44948e38e01338ba19e34ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:09.016301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1ebfb5f44948e38e01338ba19e34ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:09.017091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1ebfb5f44948e38e01338ba19e34ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:09.031630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3308/6550] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:08 2022] DELETE /identity/v3/users/9c1ebfb5f44948e38e01338ba19e34ff => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:09.036549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:09.036549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:09.036549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:09.058151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:09.059448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:09.161252 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3243/6551] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:09.183201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:09.187386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:09.309177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677027d5-be0f-44e1-950d-d90a168a1127 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:09.314741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:09.331081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:09.331081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:09.331081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:09.331081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:09.332412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:09.344569 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3309/6552] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:09.503627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:09.522107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781e68c-5259-40ba-840b-965140d41d18 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:09.525825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:09.526435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:09.526807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:09.527120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:09.530284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:09.550979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:09.552967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbb6b-bdb7-4186-b6b9-742189149a50 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:09.575807 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3244/6553] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:10.407831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:10.410142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:10.433255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cd119-49d5-4b1b-a29a-9d1045efb3fb service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:10.445868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:10.447598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:10.447882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:10.448156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] PATH_INFO: `/v3/projects/dfb55e300d9542d9a08c33731adce8af` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:10.451826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfb55e300d9542d9a08c33731adce8af)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:10.455753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:10.547624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfb55e300d9542d9a08c33731adce8af', 'request_id': 'req-f9459714-f5b0-48a8-9f6e-704559061c19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:10.555659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfb55e300d9542d9a08c33731adce8af', 'request_id': 'req-f9459714-f5b0-48a8-9f6e-704559061c19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:10.563690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfb55e300d9542d9a08c33731adce8af', 'request_id': 'req-f9459714-f5b0-48a8-9f6e-704559061c19'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:10.581385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:10.585849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] The token cache is being invalidate because project dfb55e300d9542d9a08c33731adce8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:10.587478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:10.588708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f9459714-f5b0-48a8-9f6e-704559061c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:10.592045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3310/6554] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:10 2022] DELETE /identity/v3/projects/dfb55e300d9542d9a08c33731adce8af => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:10.596974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:10.597534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:10.597534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:10.632621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:10.633850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:10.705434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:10.765637 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3245/6555] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:10.774564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:10.774878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:10.775157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:10.804067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:10.805632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:10.818574 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3311/6556] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:10.822973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:10.848529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e83beb-4288-461c-b927-82bc5c9cf25e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:10.864073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:10.864710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:10.865101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:10.865515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:10.866252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:10.866589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:10.868369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:10.887733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3246/6557] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:10.892400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:10.902272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551944bc-66fc-412d-ba42-567f39b115ee None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:10.909546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:10.910593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:10.911070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:10.911565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:10.912435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:10.915073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:10.967179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3312/6558] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:10 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:10.968495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:10.997355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99fb3e-176b-4140-aaeb-e518185a1975 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:11.155260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:11.159412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.159923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.160375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.161111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:11.167485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:11.207781 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3247/6559] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:10 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:11.210555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:11.245196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899305b-64b2-4bfb-b91e-f8cdbf5526ef admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:11.260628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:11.260890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.261134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.261358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.261962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:11.264361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:11.296007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3313/6560] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:11.300218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:11.327034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501a79-6695-4385-bcbd-82bd6ceb84c9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:11.333199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:11.334165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.335586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.335993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.336761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:11.339292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7c2d7-d556-45a8-97a3-86a54e850f43 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:11.365305 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3248/6561] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:11.369007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:11.385199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4591-a798-4a41-b2e9-c4b3d25729af admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:11.400519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:11.401209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.401491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.401771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] PATH_INFO: `/v3/projects/ef7422f951174ad894fd55b8c56077c5/users/5da20094b1d24d5ca4abe7aecdda1777/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.438454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7422f951174ad894fd55b8c56077c5, user_id=5da20094b1d24d5ca4abe7aecdda1777, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:11.448760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:11.478790 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3314/6562] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:11 2022] PUT /identity/v3/projects/ef7422f951174ad894fd55b8c56077c5/users/5da20094b1d24d5ca4abe7aecdda1777/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:11.491914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.492200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.492738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.531977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] MFA Rules not processed for user `5da20094b1d24d5ca4abe7aecdda1777`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:11.533304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:11.680160 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3249/6563] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:11.706179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:11.708404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:11.869137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613be4-a98e-478a-87b7-8912ada2893c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:11.880536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:11.881422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:11.882519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:11.883683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:11.905171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:11.908165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ff300-758b-44f7-a5d4-f9003cef6780 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:11.938104 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3315/6564] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:12.587106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:12.649947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:12.691798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2981c-6b65-4f69-ba4c-2de0bf4f2fc5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:12.699755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:12.701546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:12.702028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:12.702595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] PATH_INFO: `/v3/users/5da20094b1d24d5ca4abe7aecdda1777` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:12.703865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da20094b1d24d5ca4abe7aecdda1777)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:12.706140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:12.760044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da20094b1d24d5ca4abe7aecdda1777'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:12.761276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da20094b1d24d5ca4abe7aecdda1777'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:12.772767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da20094b1d24d5ca4abe7aecdda1777'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:12.788053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da20094b1d24d5ca4abe7aecdda1777'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:12.789508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da20094b1d24d5ca4abe7aecdda1777'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:12.843265 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3250/6565] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:12 2022] DELETE /identity/v3/users/5da20094b1d24d5ca4abe7aecdda1777 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:12.845100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:12.845560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:12.845654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:12.873687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:12.876352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:13.013284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3316/6566] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:13.077734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:13.081283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:13.272257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f2f39-d100-46c8-a672-418c77a430e0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:13.283170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:13.285514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:13.286035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:13.286540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:13.303960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:13.306399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd826f3-f2e1-41bf-be28-cb5528f29deb service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:13.336030 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3251/6567] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:14.458735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:14.461449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:14.484174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87590289-b13c-4cd7-a298-6ff2f87c866f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:14.496614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:14.500108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:14.505719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:14.506385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] PATH_INFO: `/v3/projects/ef7422f951174ad894fd55b8c56077c5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:14.508576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7422f951174ad894fd55b8c56077c5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:14.515848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:14.617827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:14.664448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7422f951174ad894fd55b8c56077c5', 'request_id': 'req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:14.680559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7422f951174ad894fd55b8c56077c5', 'request_id': 'req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:14.693586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7422f951174ad894fd55b8c56077c5', 'request_id': 'req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:14.733317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] The token cache is being invalidate because project ef7422f951174ad894fd55b8c56077c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:14.733783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:14.735667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4cf42c9a-9ada-4cdf-85aa-877a1a0227d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:14.741273 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3317/6568] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:14 2022] DELETE /identity/v3/projects/ef7422f951174ad894fd55b8c56077c5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:14.743101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:14.743578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:14.744136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:14.804219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:14.807784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:14.990295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:15.039706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3252/6569] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:15.045615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.045963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.046348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.086562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:15.090999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:15.111700 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3318/6570] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:15.116203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.130212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7dcda9-902a-4445-8f3b-e2ebcb84ca93 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.141927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.141927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.142743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.144603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.145404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:15.145825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.149354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.197444 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3253/6571] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:15.201726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.211787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a76a0c-b06b-4d4e-84a4-22b035a99d08 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.227008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.227999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.228382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.228758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.229505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.231803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.295759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3319/6572] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:15 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:15.300532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.314282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6cbe-851f-42b7-8b7e-eeffda9fe9a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.457349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.458152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.458567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.458965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.459725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.462373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.513123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.520594 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3254/6573] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:15 2022] POST /identity/v3/users => generated 284 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:15.524548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd3225-81a7-4f53-9d2b-f602a35b6245 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.534997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.535791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.536175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.536552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.537297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.539024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.569469 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3320/6574] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:15.573135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.593893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97bd71-6cb8-4d24-a4ed-37209feeec15 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.603843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.604578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.607081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.607787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.608603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.610346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4189ae-0ba3-4b36-9bd5-95d210f32fdd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.637263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3255/6575] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:15.641801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:15.660510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bce7e-5279-4030-af3f-147ac8fbc210 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:15.675448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:15.676840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.677320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.677966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] PATH_INFO: `/v3/projects/c3c7f7d8fba2426dbd63448339a6d70a/users/5a776eab9091412090141218e9bf0bf5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.714776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c7f7d8fba2426dbd63448339a6d70a, user_id=5a776eab9091412090141218e9bf0bf5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:15.720063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:15.773158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3321/6576] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:15 2022] PUT /identity/v3/projects/c3c7f7d8fba2426dbd63448339a6d70a/users/5a776eab9091412090141218e9bf0bf5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:15.791158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:15.791653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:15.791921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:15.865616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] MFA Rules not processed for user `5a776eab9091412090141218e9bf0bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:15.867441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:15.976114 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3256/6577] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:16.014838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:16.016793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:16.277258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce361a4-a50d-4d50-9d6a-6b61ceec6131 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:16.291253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:16.291884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:16.292155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:16.292427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:16.330348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:16.333095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c0e4d-8011-4bea-b211-8d8898d0e6f1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:16.354000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3322/6578] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:17.974475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:17.976824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:18.044791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:18.087121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3cb2c-91f1-4784-a174-deb30b5fb55a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.094736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.095578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.095868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.096145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] PATH_INFO: `/v3/users/efd14950096140a091067e200301ae3e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.097914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd14950096140a091067e200301ae3e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.101347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.196746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd14950096140a091067e200301ae3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:18.197441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd14950096140a091067e200301ae3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:18.216721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd14950096140a091067e200301ae3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:18.238074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd14950096140a091067e200301ae3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:18.238855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f03c2572-fede-4601-9565-26161e59d52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd14950096140a091067e200301ae3e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:18.274189 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3257/6579] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:17 2022] DELETE /identity/v3/users/efd14950096140a091067e200301ae3e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:18.279126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.279126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.279688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.297396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.297684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.297950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.340144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:18.343762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:18.376813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] MFA Rules not processed for user `2876ced64d6344e4a4be30a9b09f88d1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:18.379155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:18.489510 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3323/6580] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:18.506347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.507790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:18.517911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:18.524772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3258/6581] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:18.530847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.533038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:18.563407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d7314-8f0e-4c61-8b35-7fa27c5e1380 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.573560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.575552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.576145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.576880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.578257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.581507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.604774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade59e3-6e85-4a91-8f8d-95a73456869e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.609557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.610477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.610930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.611303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.623791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.625567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.632825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3259/6582] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:18 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:18.636385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.645065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3324/6583] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:18.695827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:18.783837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960a3d6-7de7-4c0e-8bb9-33f79aaf75a1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.793678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.795368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.797253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.797766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.798900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.801364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.842563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3260/6584] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:18 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:18.844956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.856762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc867-ad84-4f8f-8407-c37102e1af39 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.862036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.862981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.863492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.864141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.864974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.866801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.880758 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3325/6585] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:18.885114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.900918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2755-0a74-48a4-94df-5931a93da821 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:18.909092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:18.911587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:18.913006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:18.913487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] PATH_INFO: `/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7/users/d886993aa3334f53a8e3c2b1a5a00920/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:18.941219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba5880f95e643b1bbad5e173dcf49c7, user_id=d886993aa3334f53a8e3c2b1a5a00920, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:18.946682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:18.975460 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3261/6586] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:18 2022] PUT /identity/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7/users/d886993aa3334f53a8e3c2b1a5a00920/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:18.980334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:18.994161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36760e8-baaf-4185-9b27-e60e0c2b4068 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:19.078921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:19.081700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:19.133674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:19.134743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.135523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.135523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.139323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:19.142814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:19.180708 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3326/6587] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:19.190507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:19.211520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d4a1c-5aca-486c-be8c-f744684d136e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:19.222872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:19.223893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.224451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.225009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] PATH_INFO: `/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7/users/d886993aa3334f53a8e3c2b1a5a00920/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.228455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba5880f95e643b1bbad5e173dcf49c7, user_id=d886993aa3334f53a8e3c2b1a5a00920, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:19.231962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704803a-9115-428e-bb58-eb4a1051573d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:19.242748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b8b92-9b62-4833-b84d-e42cf67907d4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:19.251610 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3327/6588] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:19 2022] PUT /identity/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7/users/d886993aa3334f53a8e3c2b1a5a00920/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:19.328287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:19.329095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.329478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.329943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] PATH_INFO: `/v3/projects/b37d55810dcd46a69b760a658444b3b7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.344314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37d55810dcd46a69b760a658444b3b7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:19.346275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:19.453458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:19.473944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37d55810dcd46a69b760a658444b3b7', 'request_id': 'req-796cd576-5781-42b1-865e-bdc900e6f41e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:19.483275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37d55810dcd46a69b760a658444b3b7', 'request_id': 'req-796cd576-5781-42b1-865e-bdc900e6f41e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:19.501288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37d55810dcd46a69b760a658444b3b7', 'request_id': 'req-796cd576-5781-42b1-865e-bdc900e6f41e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:19.523280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] The token cache is being invalidate because project b37d55810dcd46a69b760a658444b3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:19.523997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:19.524803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-796cd576-5781-42b1-865e-bdc900e6f41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:19.527094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3262/6589] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:19 2022] DELETE /identity/v3/projects/b37d55810dcd46a69b760a658444b3b7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:19.532155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.532649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.532777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.571973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:19.574267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:19.701199 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3328/6590] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:19.710345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.711045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.711490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.774901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:19.776416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:19.791473 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3263/6591] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:19.793984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:19.808605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5c9e8-f95f-41b3-a5e2-7f6143a93e06 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:19.816247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:19.817184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.817767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.818350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.819840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:19.820339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:19.823052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:19.864122 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3329/6592] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:19.864656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:19.882912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4ecd1-7e05-434b-bf73-5e14cc3088fc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:19.896565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:19.898654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:19.899892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:19.900180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:19.901462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:19.908194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:19.989889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3264/6593] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:19 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:19.991847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:20.000512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:20.005875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8dcb4f-a2b8-4991-87c9-ff9d476e2f37 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:20.097363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:20.098415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.099017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.099677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.100767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:20.103847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:20.156151 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3330/6594] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:19 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:20.158287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:20.176884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63401862-0420-48f8-b5c5-1816edba13f1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:20.195729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:20.196733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.197252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.197927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.198883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:20.201868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:20.223641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3265/6595] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:20.227835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:20.243578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69ec0-9c74-4db0-9a86-01a9986532e9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:20.256127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:20.257522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.258168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.258777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.259757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:20.261940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61018a4a-3a8d-4c6b-b749-4719cf0c2bfb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:20.298288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3331/6596] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:20.300162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:20.312109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05812a-4f50-4534-bb4a-3a11edcce4ae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:20.319138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:20.320193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.320796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.321974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] PATH_INFO: `/v3/projects/0de43c8cf2764a8793b79f3a08770dbd/users/d3c82e81b54241b59b976a6f71e10135/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.343651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de43c8cf2764a8793b79f3a08770dbd, user_id=d3c82e81b54241b59b976a6f71e10135, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:20.348709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:20.379994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3266/6597] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:20 2022] PUT /identity/v3/projects/0de43c8cf2764a8793b79f3a08770dbd/users/d3c82e81b54241b59b976a6f71e10135/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:20.382407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.382759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.383034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.447298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] MFA Rules not processed for user `d3c82e81b54241b59b976a6f71e10135`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:20.448555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:20.651174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:20.651851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3332/6598] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:20.875882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc745ff5-7b7f-49ef-943f-aded135fbe67 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:20.892435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:20.896044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:20.896508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:20.897524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:20.901143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:20.905305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:20.983630 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3267/6599] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:20 2022] POST /identity/v3/projects => generated 371 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:20.987273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.008033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82509ee8-f6dd-46b3-8e95-03cea8400cb8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:21.152627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:21.153857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.154194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.154477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.155070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:21.157054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:21.203404 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3333/6600] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:20 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:21.204812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.231831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e71456-db3d-41b9-855e-587b8eb9da87 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:21.247161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:21.247859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.248174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.248378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.254717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:21.260462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:21.288775 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3268/6601] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:21.292946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.299791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:21.308252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698590a-662e-4522-8c02-eccaabdb543b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:21.319095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:21.319870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.320232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.320829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] PATH_INFO: `/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7/users/1d0bf2845b914df98248cdf1a1b104ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.339802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3e4381b98f4507bbe79b03b50ca1a7, user_id=1d0bf2845b914df98248cdf1a1b104ce, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:21.342953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:21.367628 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3334/6602] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:21 2022] PUT /identity/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7/users/1d0bf2845b914df98248cdf1a1b104ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:21.373008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.390819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074842a4-7e1f-43ed-a4a8-eac481e6fb48 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:21.485594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:21.532868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:21.533519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.533798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.534071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.534742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:21.536361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475571-8bd0-4475-8873-03af30d7ea2d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:21.553332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3269/6603] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:21.555615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.578545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3290-c023-4dce-a60c-41751108cca7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:21.588030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:21.589044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.666754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.666754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] PATH_INFO: `/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7/users/1d0bf2845b914df98248cdf1a1b104ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.666754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3e4381b98f4507bbe79b03b50ca1a7, user_id=1d0bf2845b914df98248cdf1a1b104ce, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:21.666754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:21.666754 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3335/6604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:21 2022] PUT /identity/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7/users/1d0bf2845b914df98248cdf1a1b104ce/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:21.667838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:21.667838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:21.667838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:21.708120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] MFA Rules not processed for user `1d0bf2845b914df98248cdf1a1b104ce`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:21.712007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:21.951675 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3270/6605] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:21.961568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:21.963486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:22.108971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e645da-7568-4357-8c5a-fb49730f6219 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:22.122431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:22.123714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:22.124293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:22.124391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:22.143462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:22.149225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:22.171350 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3336/6606] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:22.338640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:22.355807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c3633-5cac-4242-bf9b-ed513b172ef3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:22.366255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:22.367058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:22.367237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:22.367460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:22.669083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:22.671029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe68923-da70-49c4-b0ca-d4da84bb8456 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:22.691590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3271/6607] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:27.210104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:27.211606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:27.368774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691772a6-f201-49f6-a778-c4c698f307a2 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:27.385028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:27.387849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:27.389623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:27.390120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] PATH_INFO: `/v3/users/571351198eec43faa7a3b1abdde0c9e0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:27.398795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=571351198eec43faa7a3b1abdde0c9e0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:27.402000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:27.506208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571351198eec43faa7a3b1abdde0c9e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:27.507082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571351198eec43faa7a3b1abdde0c9e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:27.546106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571351198eec43faa7a3b1abdde0c9e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:27.595391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:27.599547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571351198eec43faa7a3b1abdde0c9e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:27.600209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571351198eec43faa7a3b1abdde0c9e0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:27.623989 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3337/6608] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:27 2022] DELETE /identity/v3/users/571351198eec43faa7a3b1abdde0c9e0 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:27.624346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312887-6175-4d1c-b308-74e18d248639 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:27.624813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312887-6175-4d1c-b308-74e18d248639 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:27.624884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80312887-6175-4d1c-b308-74e18d248639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:27.670712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-80312887-6175-4d1c-b308-74e18d248639 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:27.671946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-80312887-6175-4d1c-b308-74e18d248639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:27.783609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:27.805552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3272/6609] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:27.829897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:27.832126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:28.030127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ec44-a35e-4cf5-84eb-0fb638040d4b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:28.037572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:28.038287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:28.039507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:28.039782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:28.055870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:28.061652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b1187-8499-48dd-8288-901f9e6cc21f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:28.078529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3338/6610] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:28.506870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80312887-6175-4d1c-b308-74e18d248639 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:28.508385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-80312887-6175-4d1c-b308-74e18d248639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:28.526050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80312887-6175-4d1c-b308-74e18d248639 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:28.535782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:28.536739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:28.537195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:28.537640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] PATH_INFO: `/v3/users/d3c82e81b54241b59b976a6f71e10135` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:28.539163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c82e81b54241b59b976a6f71e10135)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:28.541363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:28.625722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c82e81b54241b59b976a6f71e10135'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:28.627292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c82e81b54241b59b976a6f71e10135'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:28.649556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c82e81b54241b59b976a6f71e10135'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:28.658277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c82e81b54241b59b976a6f71e10135'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:28.659058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c82e81b54241b59b976a6f71e10135'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:28.679746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3273/6611] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:28 2022] DELETE /identity/v3/users/d3c82e81b54241b59b976a6f71e10135 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:28.686320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:28.686891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:28.686961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:28.709387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:28.710983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:28.844759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3339/6612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:28.858915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:28.862931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:28.880237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba5ce0-ff08-416c-8f32-924002fc5bae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:28.892177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:28.893419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:28.893850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:28.894279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] PATH_INFO: `/v3/projects/3329d177ef8e40d3b511b4656ba925a3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:28.896582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3329d177ef8e40d3b511b4656ba925a3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:28.901169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:28.906431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:28.909196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.045432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3329d177ef8e40d3b511b4656ba925a3', 'request_id': 'req-4abec667-3129-438c-909e-ef0c4ca6fdc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:29.054841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0873c1-2c63-48ba-af65-d0ba17dc3bc5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:29.080711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3329d177ef8e40d3b511b4656ba925a3', 'request_id': 'req-4abec667-3129-438c-909e-ef0c4ca6fdc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:29.111798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3329d177ef8e40d3b511b4656ba925a3', 'request_id': 'req-4abec667-3129-438c-909e-ef0c4ca6fdc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:29.139455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] The token cache is being invalidate because project 3329d177ef8e40d3b511b4656ba925a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:29.139958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:29.141572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4abec667-3129-438c-909e-ef0c4ca6fdc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:29.147370 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3274/6613] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:28 2022] DELETE /identity/v3/projects/3329d177ef8e40d3b511b4656ba925a3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:29.149782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1414-8b95-4184-ac29-9ae6fe5ef4ed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.151297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1414-8b95-4184-ac29-9ae6fe5ef4ed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.152019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1414-8b95-4184-ac29-9ae6fe5ef4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.197561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-522c1414-8b95-4184-ac29-9ae6fe5ef4ed None None] MFA Rules not processed for user `c1acf8d4b1384c8c8d7014bd8baf897e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:29.200181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-522c1414-8b95-4184-ac29-9ae6fe5ef4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.204028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:29.223207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:29.223847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.224129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.224400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.229524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.354037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:29.374948 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3275/6614] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:29.377065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.377350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.377620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.385372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:29.387600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3456055-140f-45e6-ab53-8224906d147b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:29.426854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3340/6615] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:29.443849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:29.445407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.458728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3276/6616] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:29.467245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.468066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.468652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.514278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:29.515716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.542953 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3341/6617] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:29.546752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:29.568954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91affabb-3e95-4d16-8bbe-03207c346b9f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:29.583704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:29.584523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.585028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.585397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.586249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:29.587565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:29.593219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:29.614197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3277/6618] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:29.615676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:29.628348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab78ff-5ba6-437e-b5fd-99257e7807ca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:29.638294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:29.639643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.639929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.642384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.643117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:29.645070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76551b-9faf-488d-b09b-f317f455afe1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:29.714044 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3342/6619] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/projects => generated 357 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:29.716809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:29.749458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0ae6e-d22a-4bdf-b677-9cbe937a2f1c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:29.802877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.803259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.803546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.845291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] MFA Rules not processed for user `d886993aa3334f53a8e3c2b1a5a00920`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:29.846612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:29.894385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:29.895256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.895682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.896134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.896773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:29.899070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:29.962635 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3278/6620] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/users => generated 279 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:29.968492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:29.984528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c91be9-4c2d-45d7-890c-b8ebe0a43f1f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:29.991546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:29.992017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:29.992239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:29.992739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:29.993152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:29.995606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:30.001694 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3343/6621] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:30.016025 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3279/6622] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:30.020206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.069011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae07bf-9002-4932-814f-089b96d5551d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:30.071872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.075422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.092054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:30.093021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:30.093411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:30.094267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:30.095036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:30.096904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:30.123943 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3344/6623] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:30.125060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.138224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5db3a5-5fbc-4484-a1bc-bcdeab31961f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:30.164536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:30.165454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:30.165908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:30.166448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] PATH_INFO: `/v3/projects/b8587c7e1fb24dc684bf5faa899f0761/users/cf091205faca492896ba147af407a0c3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:30.197577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8587c7e1fb24dc684bf5faa899f0761, user_id=cf091205faca492896ba147af407a0c3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:30.201097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bca399-febb-4027-a655-2f97204df81f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:30.239835 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3345/6624] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:30 2022] PUT /identity/v3/projects/b8587c7e1fb24dc684bf5faa899f0761/users/cf091205faca492896ba147af407a0c3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:30.249600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:30.249600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:30.249600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:30.265311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8318cd-2fd7-416f-928b-f8cbb6a5051b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:30.354393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] MFA Rules not processed for user `cf091205faca492896ba147af407a0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:30.360729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.376520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:30.403233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:30.403882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:30.404166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:30.404443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:30.406356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.473664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:30.540021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:30.541894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:30.548001 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3346/6625] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:30.549607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.554208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.554873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3280/6626] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:30.677408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.679329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.742954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5abac-17f2-418c-8674-9b36ab8b3e16 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:30.750554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:30.750554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:30.750554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:30.750554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] PATH_INFO: `/v3/projects/0de43c8cf2764a8793b79f3a08770dbd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:30.754733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de43c8cf2764a8793b79f3a08770dbd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:30.756689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:30.849878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb3ff-8996-4d48-9371-25cc0cd41f18 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:30.864908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de43c8cf2764a8793b79f3a08770dbd', 'request_id': 'req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:30.883416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de43c8cf2764a8793b79f3a08770dbd', 'request_id': 'req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:30.897759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de43c8cf2764a8793b79f3a08770dbd', 'request_id': 'req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:30.925623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] The token cache is being invalidate because project 0de43c8cf2764a8793b79f3a08770dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:30.925918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:30.926738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:30.928451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3347/6627] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:30 2022] DELETE /identity/v3/projects/0de43c8cf2764a8793b79f3a08770dbd => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:30.932459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:30.934736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:30.994455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:31.029892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9078-c50b-44ee-82c2-52f1a3c735ec admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:31.032289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:31.033165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:31.033610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:31.034117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:31.036469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:31.037434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:31.038201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:31.038612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:31.039098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] PATH_INFO: `/v3/users/1d0bf2845b914df98248cdf1a1b104ce` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:31.041583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0bf2845b914df98248cdf1a1b104ce)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:31.048574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:31.172314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0bf2845b914df98248cdf1a1b104ce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.173108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0bf2845b914df98248cdf1a1b104ce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.185136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0bf2845b914df98248cdf1a1b104ce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.190903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:31.192895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:31.215684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3281/6628] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:31.225336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0bf2845b914df98248cdf1a1b104ce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.227345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2c0cd173-a5ed-4f71-9d59-609976b30db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0bf2845b914df98248cdf1a1b104ce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.252304 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3348/6629] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:30 2022] DELETE /identity/v3/users/1d0bf2845b914df98248cdf1a1b104ce => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:31.769089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:31.783528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5ca2-eb7a-46ce-9a46-a08e052574ac service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:31.851391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:31.875943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:31.877139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:31.877512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:31.877901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] PATH_INFO: `/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:31.879321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3e4381b98f4507bbe79b03b50ca1a7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:31.881478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:31.942937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3e4381b98f4507bbe79b03b50ca1a7', 'request_id': 'req-48516972-e18b-4f88-8d0f-8b3840d17fb8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.951798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3e4381b98f4507bbe79b03b50ca1a7', 'request_id': 'req-48516972-e18b-4f88-8d0f-8b3840d17fb8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.963316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3e4381b98f4507bbe79b03b50ca1a7', 'request_id': 'req-48516972-e18b-4f88-8d0f-8b3840d17fb8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.987592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] The token cache is being invalidate because project 6e3e4381b98f4507bbe79b03b50ca1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:31.989537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:31.990817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48516972-e18b-4f88-8d0f-8b3840d17fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:32.002679 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3282/6630] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:31 2022] DELETE /identity/v3/projects/6e3e4381b98f4507bbe79b03b50ca1a7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:32.003297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.004119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.004399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.047694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:32.051108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:32.223340 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3349/6631] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:32.232563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.232856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.233143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.282104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:32.284328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:32.346142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3283/6632] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:32.350860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:32.373141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0895a-0247-47e7-999b-d295ee31178e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:32.386257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:32.390923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.391940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.392324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.395470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:32.397345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:32.399947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:32.445478 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3350/6633] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:32.446849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:32.458331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019fc9f-0ff7-45d0-aae5-775cfa2ef6f3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:32.476690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:32.477620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.477957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.486085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.487089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:32.491245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:32.573115 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3284/6634] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:32 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:32.576754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:32.594481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa505e-2f2b-4077-a51a-fd6a96fd5573 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:32.734436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:32.765506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:32.766427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.766911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.767199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.767979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:32.772930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:32.844951 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3351/6635] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:32 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:32.849115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:32.865319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03579b11-ef1a-4868-991d-d1aadb85aead admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:32.885683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:32.886193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.888518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.889240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.890379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:32.895659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:32.934982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:32.935839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3285/6636] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:32.953886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01f913-2101-45d7-bea4-a7f645b55c0a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:32.980703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:32.985750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:32.986096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:32.986407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:32.987409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:32.993571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf641e1-28b6-4181-a6dc-63f69f88743d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:33.041259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3352/6637] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:33.042570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:33.070139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140245b5-d611-4e01-b439-24f1bd450705 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:33.124975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:33.125658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:33.126011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:33.126204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] PATH_INFO: `/v3/projects/086b4c29376f4766a71662e7bb6c1a2b/users/d6b727aaf63e4d5cb7444d53c90cfacc/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:33.197031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=086b4c29376f4766a71662e7bb6c1a2b, user_id=d6b727aaf63e4d5cb7444d53c90cfacc, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:33.206971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:33.254338 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3286/6638] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:33 2022] PUT /identity/v3/projects/086b4c29376f4766a71662e7bb6c1a2b/users/d6b727aaf63e4d5cb7444d53c90cfacc/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:33.265454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:33.265768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:33.266031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:33.419889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] MFA Rules not processed for user `d6b727aaf63e4d5cb7444d53c90cfacc`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:33.421154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:33.680347 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3353/6639] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:33.685753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:33.855713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4528a7-aeae-43a1-952f-fb008f57b7e2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:33.870437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:33.871969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:33.872296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:33.872584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:33.873258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:33.875243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:33.905636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:33.970254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3287/6640] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:33 2022] POST /identity/v3/projects => generated 363 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:33.974566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:34.003967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ffd1b-1a5d-4f58-b7fc-42f13281d252 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:34.198571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:34.200188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.200188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.200343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.200987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:34.222934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:34.290550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3354/6641] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:33 2022] POST /identity/v3/users => generated 288 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:34.292426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:34.322633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dad61-f399-4d6a-8f45-1becf34b742f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:34.343594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:34.345300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.347199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.347747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.358718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:34.363136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:34.382721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:34.383458 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3288/6642] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:34.398769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799390f-62e9-4564-99fd-e5979d24ec86 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:34.413485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:34.414416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.414745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.415025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] PATH_INFO: `/v3/projects/b30b158c43dd411db8be4ecd76bfb52e/users/dce48eb23dbd41d29fcea03a06a7b809/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.454750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30b158c43dd411db8be4ecd76bfb52e, user_id=dce48eb23dbd41d29fcea03a06a7b809, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:34.460561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:34.490860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3355/6643] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:34 2022] PUT /identity/v3/projects/b30b158c43dd411db8be4ecd76bfb52e/users/dce48eb23dbd41d29fcea03a06a7b809/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:34.495930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:34.508380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1fb5c-d6dd-429d-ae88-460d03c3dca4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:34.623631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:34.625901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.627698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.628391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.629518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:34.633940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663b1d-3a35-48c7-b27c-9d58b5ace9e5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:34.666968 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3289/6644] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:34.669969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:34.686522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f304a-3379-4875-8312-22e3a2b0ed29 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:34.698010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:34.700923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.701395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.701826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] PATH_INFO: `/v3/projects/b30b158c43dd411db8be4ecd76bfb52e/users/dce48eb23dbd41d29fcea03a06a7b809/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.711505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30b158c43dd411db8be4ecd76bfb52e, user_id=dce48eb23dbd41d29fcea03a06a7b809, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:34.721284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:34.759621 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3356/6645] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:34 2022] PUT /identity/v3/projects/b30b158c43dd411db8be4ecd76bfb52e/users/dce48eb23dbd41d29fcea03a06a7b809/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:34.776281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:34.776624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:34.776915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:34.895236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] MFA Rules not processed for user `dce48eb23dbd41d29fcea03a06a7b809`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:34.897348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:35.080634 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3290/6646] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:35.096126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:35.098744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:35.108907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:35.216497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8258-fa37-41ab-8aab-0c5aa88bbfd9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:35.239412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:35.240714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:35.241192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:35.241607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:35.256614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:35.258627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf3ec2-1bcd-404a-b531-44f3b4cd1b18 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:35.281875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3357/6647] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:36.239131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:36.410276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7c488-68ba-4a23-b0cc-5ebc2fceb049 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:36.424508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:36.425285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:36.425658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:36.426170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] PATH_INFO: `/v3/users/d6b727aaf63e4d5cb7444d53c90cfacc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:36.429219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b727aaf63e4d5cb7444d53c90cfacc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:36.431629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:36.448319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:36.525310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b727aaf63e4d5cb7444d53c90cfacc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:36.526537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b727aaf63e4d5cb7444d53c90cfacc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:36.542170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b727aaf63e4d5cb7444d53c90cfacc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:36.554067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b727aaf63e4d5cb7444d53c90cfacc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:36.555157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b727aaf63e4d5cb7444d53c90cfacc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:36.592687 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3291/6648] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:36 2022] DELETE /identity/v3/users/d6b727aaf63e4d5cb7444d53c90cfacc => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:36.596912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:36.598207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:36.598946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:36.638920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:36.640542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:36.797007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3358/6649] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:36.811178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:36.823314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:36.965252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6ecb-2b0f-464b-91da-79e36c908bb1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:36.974256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:36.975141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:36.975438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:36.975917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:37.011162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:37.015765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:37.038464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3292/6650] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:37.950971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:37.952666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:37.973482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e8d00-d92d-4e69-b6fc-0fd1cfe4d822 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:37.987010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:37.987935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:37.988287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:37.988721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] PATH_INFO: `/v3/projects/086b4c29376f4766a71662e7bb6c1a2b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:37.992623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=086b4c29376f4766a71662e7bb6c1a2b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:37.998133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:38.111122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '086b4c29376f4766a71662e7bb6c1a2b', 'request_id': 'req-88a8ecef-a381-48f6-86ff-371278a7a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.156287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '086b4c29376f4766a71662e7bb6c1a2b', 'request_id': 'req-88a8ecef-a381-48f6-86ff-371278a7a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.186798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '086b4c29376f4766a71662e7bb6c1a2b', 'request_id': 'req-88a8ecef-a381-48f6-86ff-371278a7a833'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.223543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] The token cache is being invalidate because project 086b4c29376f4766a71662e7bb6c1a2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:38.223990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.229455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.232214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3359/6651] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:37 2022] DELETE /identity/v3/projects/086b4c29376f4766a71662e7bb6c1a2b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:38.236302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:38.251002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c22e32-6bb4-44c2-b4f5-82a9eb8bdf00 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:38.452571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:38.453502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:38.454146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:38.455498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] PATH_INFO: `/v3/users/dce48eb23dbd41d29fcea03a06a7b809` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:38.459133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce48eb23dbd41d29fcea03a06a7b809)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:38.466626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:38.606979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:38.702040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce48eb23dbd41d29fcea03a06a7b809'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.710631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce48eb23dbd41d29fcea03a06a7b809'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.754375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce48eb23dbd41d29fcea03a06a7b809'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.782357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce48eb23dbd41d29fcea03a06a7b809'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.788958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce48eb23dbd41d29fcea03a06a7b809'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:38.864148 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3293/6652] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:38 2022] DELETE /identity/v3/users/dce48eb23dbd41d29fcea03a06a7b809 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:40.009247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:40.011041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.047220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ecef-a381-48f6-86ff-371278a7a833 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:40.149250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:40.244219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:40.245526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.245838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.247410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] PATH_INFO: `/v3/projects/b30b158c43dd411db8be4ecd76bfb52e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.251667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30b158c43dd411db8be4ecd76bfb52e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:40.264497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:40.363285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:40.366800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.403394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8bde-11c5-4493-ac50-6d304bcbd835 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:40.406968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30b158c43dd411db8be4ecd76bfb52e', 'request_id': 'req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.442913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30b158c43dd411db8be4ecd76bfb52e', 'request_id': 'req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.460539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30b158c43dd411db8be4ecd76bfb52e', 'request_id': 'req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.491632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] The token cache is being invalidate because project b30b158c43dd411db8be4ecd76bfb52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:40.492013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.493474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7e5bf244-7e79-4c39-a8fa-2d8dee5c2be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.504183 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3360/6653] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:40 2022] DELETE /identity/v3/projects/b30b158c43dd411db8be4ecd76bfb52e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:40.504298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af1e40-8903-4e1b-91e1-c31732b5a77a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.504776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af1e40-8903-4e1b-91e1-c31732b5a77a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.505194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af1e40-8903-4e1b-91e1-c31732b5a77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.556735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c7af1e40-8903-4e1b-91e1-c31732b5a77a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:40.561430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c7af1e40-8903-4e1b-91e1-c31732b5a77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.575560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:40.577544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.578050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.579806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] PATH_INFO: `/v3/users/5a776eab9091412090141218e9bf0bf5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.583052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a776eab9091412090141218e9bf0bf5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:40.587843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:40.690449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a776eab9091412090141218e9bf0bf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.691048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a776eab9091412090141218e9bf0bf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.708887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a776eab9091412090141218e9bf0bf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.718673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3361/6654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:40.721449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a776eab9091412090141218e9bf0bf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.722181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97c6f570-c1f8-4a73-9232-0ea54377e535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a776eab9091412090141218e9bf0bf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:40.727595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.728491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.729139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.753788 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3294/6655] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:40 2022] DELETE /identity/v3/users/5a776eab9091412090141218e9bf0bf5 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:40.757755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.757973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.758274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.763197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:40.764582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.779987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:40.781710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.801941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:40.896502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3362/6656] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:40.901101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:40.909421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:40.925138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54a9fe-1eaf-49f1-89bb-e7e9ab31ae9c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:40.940223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:40.941193 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3295/6657] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:40.951854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:40.953323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:40.955269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:40.955943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:40.956225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:40.957292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:40.957605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:40.960684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:40.982951 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3363/6658] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:14:40.983915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.002742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f93d-0216-493a-9ad1-802116540968 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.009147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.009699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.010246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.010246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.010902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.013820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.060979 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3364/6659] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:40 2022] POST /identity/v3/projects => generated 357 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:41.065041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.079784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02052d-3b6a-4b24-9a23-6bdc252d8fba None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.164831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2c22c-1cf8-4c12-b594-d49feb371bc0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.171455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.173017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.173360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.173652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.174474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.176510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.196653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.197411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.197586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.197827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.210903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3365/6660] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:41 2022] POST /identity/v3/users => generated 279 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:41.211751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.215789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.217616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.225882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac60ae-9d29-482e-b67c-355c0c8a3b37 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.232743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.233361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.233644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.233921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.234676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.236632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.239061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3296/6661] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:41.248799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3366/6662] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:41.252898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.266924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0ce38-89e1-4abf-af58-3fc5fdaacd01 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.272465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.273098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.273358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.273637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.274514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.278116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04979d0-e9d5-4fd1-99f0-ae2f4993cfa1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.290673 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3297/6663] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:41.295123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.307350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce521ec-a7b0-49fa-87d4-8e6af6b08947 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.315693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.316565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.316676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.316968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] PATH_INFO: `/v3/projects/7683d0c0cb694f4c8a4354a9ad0d8aaf/users/64f767bbf9624756afb3de46fb8f6019/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.334030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7683d0c0cb694f4c8a4354a9ad0d8aaf, user_id=64f767bbf9624756afb3de46fb8f6019, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.336804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.351963 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3367/6664] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:41 2022] PUT /identity/v3/projects/7683d0c0cb694f4c8a4354a9ad0d8aaf/users/64f767bbf9624756afb3de46fb8f6019/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:41.363756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.364617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.364898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.397622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] MFA Rules not processed for user `64f767bbf9624756afb3de46fb8f6019`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:41.399013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:41.477715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:41.486812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3298/6665] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:41.487620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.563383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8bad9-803b-462e-8cb3-9a711f267535 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.570199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.570976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.571478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.571757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.572432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.575362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.613248 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3368/6666] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:41 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:41.614294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.626346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddc10f-9d08-427a-b9e7-bcf5e2236273 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.720538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.721830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.722330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.722781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.723816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.726925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.761524 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3299/6667] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:41 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:41.763073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.775117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7da0a7-3b20-4ee7-babf-3160858ff837 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.781172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.781973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.782295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.782613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.783306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.785088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.787027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:41.801972 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3369/6668] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:41.805799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.820802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded79d-7590-4937-b40a-ca78ee83edeb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:41.826630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:41.827314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:41.827634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:41.829361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] PATH_INFO: `/v3/projects/23e49cba5d5a47148f27d75ccbed2866/users/8a865f49291a4e4b9b8f1dc2dcf28aaf/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:41.852129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e49cba5d5a47148f27d75ccbed2866, user_id=8a865f49291a4e4b9b8f1dc2dcf28aaf, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:41.855888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:41.877926 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3300/6669] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:41 2022] PUT /identity/v3/projects/23e49cba5d5a47148f27d75ccbed2866/users/8a865f49291a4e4b9b8f1dc2dcf28aaf/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:41.888898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.889510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:41.890966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:41.912503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa078efc-2582-46a3-9166-a15dd33b2168 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.023394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.024031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.025100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.025380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.026036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.027982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.054149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441f98b-837d-4847-a7fc-dc84a892de21 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.059631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3370/6670] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:42.060926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.061669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.062081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.062305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] PATH_INFO: `/v3/projects/c3c7f7d8fba2426dbd63448339a6d70a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.063499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.064308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c7f7d8fba2426dbd63448339a6d70a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.066324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.074924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87063d23-761e-4485-92dd-0f5e5224a8fe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.080871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.081703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.083359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.083748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] PATH_INFO: `/v3/projects/23e49cba5d5a47148f27d75ccbed2866/users/8a865f49291a4e4b9b8f1dc2dcf28aaf/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.086459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e49cba5d5a47148f27d75ccbed2866, user_id=8a865f49291a4e4b9b8f1dc2dcf28aaf, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.090187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d434-6e38-43f7-8174-a266d4d29c11 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.106244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3371/6671] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:42 2022] PUT /identity/v3/projects/23e49cba5d5a47148f27d75ccbed2866/users/8a865f49291a4e4b9b8f1dc2dcf28aaf/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:42.118316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7790-b824-4588-8dab-13320fc96658 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.119038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7790-b824-4588-8dab-13320fc96658 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.119451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7790-b824-4588-8dab-13320fc96658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.130362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c7f7d8fba2426dbd63448339a6d70a', 'request_id': 'req-9008feb2-5770-42f4-884a-85cf573ca71b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:42.146388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c7f7d8fba2426dbd63448339a6d70a', 'request_id': 'req-9008feb2-5770-42f4-884a-85cf573ca71b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:42.161877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c7f7d8fba2426dbd63448339a6d70a', 'request_id': 'req-9008feb2-5770-42f4-884a-85cf573ca71b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:42.164700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:42.180978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-620f7790-b824-4588-8dab-13320fc96658 None None] MFA Rules not processed for user `8a865f49291a4e4b9b8f1dc2dcf28aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:42.182553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-620f7790-b824-4588-8dab-13320fc96658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:42.184957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] The token cache is being invalidate because project c3c7f7d8fba2426dbd63448339a6d70a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:42.185293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:42.186017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9008feb2-5770-42f4-884a-85cf573ca71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:42.187694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3301/6672] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:41 2022] DELETE /identity/v3/projects/c3c7f7d8fba2426dbd63448339a6d70a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:42.193191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827596d-7422-4da9-8aa6-d2afc6a271c6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.193479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827596d-7422-4da9-8aa6-d2afc6a271c6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.193758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827596d-7422-4da9-8aa6-d2afc6a271c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.228793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8827596d-7422-4da9-8aa6-d2afc6a271c6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:42.230863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8827596d-7422-4da9-8aa6-d2afc6a271c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:42.324069 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3372/6673] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:42.334332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f7790-b824-4588-8dab-13320fc96658 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.336299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-620f7790-b824-4588-8dab-13320fc96658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:42.338751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3302/6674] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:42.348170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.348569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.348921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.376882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:42.378161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:42.389250 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3303/6675] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:42.393515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.412164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4856478-8265-456e-847f-15f3a5bc285e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.419605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.420728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.421160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.421582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.422453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:42.423014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.424994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.431394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f7790-b824-4588-8dab-13320fc96658 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.436269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.437112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.437469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.437825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.442220 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3304/6676] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:14:42.447050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.455152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.457592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.469904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383df5c-4109-4353-9c6d-325d75831715 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.470928 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3373/6677] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:42.477026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.477934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.478371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.478821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.479593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.482473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.539048 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3305/6678] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:42.542316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.553857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84089a-b4fc-4d14-b967-5318f741ed24 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.558717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:42.575286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.641229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.642753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.643273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.643739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.644569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.647678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.686028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3374/6679] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:42.689863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.703534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944e19-08c8-43b1-ae1b-fb3beee2b435 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.709020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d841b-99ed-4b0c-afb4-62b209b0ed82 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.710830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.711471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.711756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.712054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.712676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.715680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.715978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.717018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.717443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.717863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.720638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:42.733003 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3375/6680] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:42.736561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.753350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373df721-dba1-45c1-b009-f01f46e16bac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.759346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.760521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.760805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.761319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.762797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.764846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.781415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3376/6681] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:42.785763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:42.797223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30794-c46e-4d63-af37-b4448cf2dc1c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:42.803398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:42.803897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.804077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.804349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.804620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] PATH_INFO: `/v3/projects/5fe11c1d03464af48f779873c2918240/users/b33bf333a7784ae886052bb0e0ad6bd4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.805409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31cac01-ba47-41b6-9848-e13531fab273 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.820626 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3306/6682] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:42.824620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe11c1d03464af48f779873c2918240, user_id=b33bf333a7784ae886052bb0e0ad6bd4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:42.827741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:42.847898 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3377/6683] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:42 2022] PUT /identity/v3/projects/5fe11c1d03464af48f779873c2918240/users/b33bf333a7784ae886052bb0e0ad6bd4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:42.859402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:42.859663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:42.859940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:42.897637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:42.915470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] MFA Rules not processed for user `b33bf333a7784ae886052bb0e0ad6bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:42.918738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:43.053516 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3307/6684] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:43.057584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.191538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d93661-ecaa-4bbc-a885-96bf7338fdb7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.197224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.198081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.198515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.199626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.200975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.203353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.246515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3378/6685] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:43 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:43.251518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.272674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae97344-581a-47b0-8b23-014d5d679614 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.359369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.360054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.360348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.360599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.361231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.363776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.400332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3308/6686] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:43 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:43.401513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.415385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3104-a386-4fc1-9117-eb6a8bf3174c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.420652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.421519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.422194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.422687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.424190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.426127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.442079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3379/6687] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:43.446356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.457898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c33b3-dcda-4724-a0ea-e77a5f22b111 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.463083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.463793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.464922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.465341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] PATH_INFO: `/v3/projects/5c43168200b2468b8376dabbbd9892ea/users/1f2b05b3577647c1ba411903ec168434/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.483073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c43168200b2468b8376dabbbd9892ea, user_id=1f2b05b3577647c1ba411903ec168434, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.485921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.503827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3309/6688] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:43 2022] PUT /identity/v3/projects/5c43168200b2468b8376dabbbd9892ea/users/1f2b05b3577647c1ba411903ec168434/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:43.507827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.519039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523761db-4d32-4845-8ee9-6d135a62d708 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.546852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:43.606250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.606928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.607342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.607777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.608380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.610049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2698b8-dd09-475e-8a84-db89855b2669 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.626209 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3380/6689] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:43.627411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.644156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2412c-dd92-4d77-9c10-e6dde312ef9d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.653637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.655641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.656055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.656457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] PATH_INFO: `/v3/projects/5c43168200b2468b8376dabbbd9892ea/users/1f2b05b3577647c1ba411903ec168434/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.659564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c43168200b2468b8376dabbbd9892ea, user_id=1f2b05b3577647c1ba411903ec168434, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:43.662991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:43.681061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3310/6690] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:43 2022] PUT /identity/v3/projects/5c43168200b2468b8376dabbbd9892ea/users/1f2b05b3577647c1ba411903ec168434/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:43.693245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.693534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.693796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.729484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] MFA Rules not processed for user `1f2b05b3577647c1ba411903ec168434`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:43.731671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:43.814121 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3381/6691] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:43.825169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:43.828364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:43.950816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676a84-9a63-498a-825a-43fb1cdbf1fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:43.955269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:43.956064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:43.956434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:43.956843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:43.965257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:44.039790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:44.041389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c69c01-4b5b-4625-8028-e125b6bf6b30 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:44.054991 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3311/6692] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:46.037294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:46.249356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2dd09-1cbe-41a5-a2d1-6d68d925de24 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:46.255355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:46.256059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:46.256392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:46.256688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] PATH_INFO: `/v3/users/b33bf333a7784ae886052bb0e0ad6bd4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:46.267146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33bf333a7784ae886052bb0e0ad6bd4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:46.269431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:46.422939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33bf333a7784ae886052bb0e0ad6bd4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.424888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33bf333a7784ae886052bb0e0ad6bd4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.430308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:46.444381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33bf333a7784ae886052bb0e0ad6bd4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.481961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33bf333a7784ae886052bb0e0ad6bd4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.491718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33bf333a7784ae886052bb0e0ad6bd4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.518695 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3382/6693] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:46 2022] DELETE /identity/v3/users/b33bf333a7784ae886052bb0e0ad6bd4 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:46.523596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:46.524492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:46.524993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:46.551671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:46.554604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:46.686753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:46.688726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:46.733583 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3312/6694] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:46.747415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:46.749840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:46.820711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620a6c-f0b1-4338-9c73-20c829027063 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:46.835182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:46.835844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:46.836121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:46.836393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] PATH_INFO: `/v3/users/64f767bbf9624756afb3de46fb8f6019` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:46.837552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f767bbf9624756afb3de46fb8f6019)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:46.840517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:46.894145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f83c1-f0ee-4517-9fee-4c48198f1d7f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:46.899377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:46.900289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:46.900671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:46.900947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:46.921626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f767bbf9624756afb3de46fb8f6019'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.922522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f767bbf9624756afb3de46fb8f6019'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.929261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:46.931814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66a402-d455-4ff5-8cb2-67f229070db2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:46.937593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f767bbf9624756afb3de46fb8f6019'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.945936 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3313/6695] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:46.962149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f767bbf9624756afb3de46fb8f6019'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:46.963818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f767bbf9624756afb3de46fb8f6019'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.003833 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3383/6696] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:46 2022] DELETE /identity/v3/users/64f767bbf9624756afb3de46fb8f6019 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:47.008153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:47.008583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:47.009286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:47.036304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:47.037659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:47.108490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:47.134459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3314/6697] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:47.155933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:47.159805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:47.222921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:47.250279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376fa71-7332-4634-8cb1-22bdce887a8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:47.254670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:47.255535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:47.256159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:47.256617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:47.268502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:47.270568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:47.283990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3384/6698] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:47.318188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:47.319771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:47.332984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb7661-96c8-4f2f-9642-ffceb0320f32 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:47.348225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:47.348907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:47.349595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:47.349595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] PATH_INFO: `/v3/projects/5fe11c1d03464af48f779873c2918240` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:47.350821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe11c1d03464af48f779873c2918240)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:47.353672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:47.423991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe11c1d03464af48f779873c2918240', 'request_id': 'req-5447d938-effc-4abc-9a81-f3ca7e76f259'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.430978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe11c1d03464af48f779873c2918240', 'request_id': 'req-5447d938-effc-4abc-9a81-f3ca7e76f259'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.440944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe11c1d03464af48f779873c2918240', 'request_id': 'req-5447d938-effc-4abc-9a81-f3ca7e76f259'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.453604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] The token cache is being invalidate because project 5fe11c1d03464af48f779873c2918240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:47.453954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.454674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.456436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3315/6699] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:47 2022] DELETE /identity/v3/projects/5fe11c1d03464af48f779873c2918240 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:47.460256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:47.471771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73b4d2-db73-43dc-b05d-44ed11a0a080 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:47.535798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:47.536445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:47.536901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:47.537597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] PATH_INFO: `/v3/users/1f2b05b3577647c1ba411903ec168434` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:47.539954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2b05b3577647c1ba411903ec168434)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:47.542769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:47.675926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2b05b3577647c1ba411903ec168434'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.676876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2b05b3577647c1ba411903ec168434'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.687306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2b05b3577647c1ba411903ec168434'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.718732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2b05b3577647c1ba411903ec168434'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.727731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2b05b3577647c1ba411903ec168434'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:47.747141 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3385/6700] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:47 2022] DELETE /identity/v3/users/1f2b05b3577647c1ba411903ec168434 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:48.259435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:48.261884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:48.278178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d938-effc-4abc-9a81-f3ca7e76f259 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:48.355584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:48.356339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:48.356721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:48.357272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] PATH_INFO: `/v3/projects/7683d0c0cb694f4c8a4354a9ad0d8aaf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:48.358883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7683d0c0cb694f4c8a4354a9ad0d8aaf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:48.372188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:48.435401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7683d0c0cb694f4c8a4354a9ad0d8aaf', 'request_id': 'req-b6852fff-4ec4-4670-afd0-5ee7b8133db0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.441351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7683d0c0cb694f4c8a4354a9ad0d8aaf', 'request_id': 'req-b6852fff-4ec4-4670-afd0-5ee7b8133db0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.449667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7683d0c0cb694f4c8a4354a9ad0d8aaf', 'request_id': 'req-b6852fff-4ec4-4670-afd0-5ee7b8133db0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.468067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] The token cache is being invalidate because project 7683d0c0cb694f4c8a4354a9ad0d8aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:48.468067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.468855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.470783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3316/6701] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:48 2022] DELETE /identity/v3/projects/7683d0c0cb694f4c8a4354a9ad0d8aaf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:48.474936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:48.476526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:48.499520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168e3cd-87f9-4ce4-a246-98c727f0e94a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:48.578958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:48.579450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:48.579759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:48.580131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] PATH_INFO: `/v3/users/8a865f49291a4e4b9b8f1dc2dcf28aaf` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:48.582366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a865f49291a4e4b9b8f1dc2dcf28aaf)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:48.584653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:48.587019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:48.589465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:48.601604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:48.616490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852fff-4ec4-4670-afd0-5ee7b8133db0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:48.617515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:48.643700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:48.644393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:48.644628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:48.650968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] PATH_INFO: `/v3/projects/5c43168200b2468b8376dabbbd9892ea` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:48.669742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c43168200b2468b8376dabbbd9892ea)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:48.669742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:48.707853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a865f49291a4e4b9b8f1dc2dcf28aaf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.708950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a865f49291a4e4b9b8f1dc2dcf28aaf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.722922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a865f49291a4e4b9b8f1dc2dcf28aaf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.750086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c43168200b2468b8376dabbbd9892ea', 'request_id': 'req-d34538b9-d8c5-407e-80a3-07c46da10f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.757997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c43168200b2468b8376dabbbd9892ea', 'request_id': 'req-d34538b9-d8c5-407e-80a3-07c46da10f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.770099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c43168200b2468b8376dabbbd9892ea', 'request_id': 'req-d34538b9-d8c5-407e-80a3-07c46da10f9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.775193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a865f49291a4e4b9b8f1dc2dcf28aaf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.776083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61909348-6a67-4d57-a679-35eadc59fa19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a865f49291a4e4b9b8f1dc2dcf28aaf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.786347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] The token cache is being invalidate because project 5c43168200b2468b8376dabbbd9892ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:48.786676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.787342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:48.790889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3317/6702] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:48 2022] DELETE /identity/v3/projects/5c43168200b2468b8376dabbbd9892ea => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:48.790889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:48.791559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:48.796206 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3386/6703] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:48 2022] DELETE /identity/v3/users/8a865f49291a4e4b9b8f1dc2dcf28aaf => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:48.806977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948eb6-f951-44f6-934c-378d28d9f780 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:48.807434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948eb6-f951-44f6-934c-378d28d9f780 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:48.807886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07948eb6-f951-44f6-934c-378d28d9f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:48.835002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-07948eb6-f951-44f6-934c-378d28d9f780 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:48.836341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07948eb6-f951-44f6-934c-378d28d9f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:48.923683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34538b9-d8c5-407e-80a3-07c46da10f9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:48.971914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3387/6704] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:48.974014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:48.974300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:48.974645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.013927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:49.015279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.053767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.054598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.054955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.055236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] PATH_INFO: `/v3/users/cf091205faca492896ba147af407a0c3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.056426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf091205faca492896ba147af407a0c3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.059241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.082357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3388/6705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:49.085209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.099905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8b055-dde2-4881-8ecb-78526d619ea3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.106732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.107612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.107939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.108236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.109018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:49.109885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.111133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.129756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3389/6706] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:49.131567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.146431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353cc3a-771d-4058-95b9-143c757a7824 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.148680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:49.156879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.159366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.159366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.159571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.161335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.163570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.169761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf091205faca492896ba147af407a0c3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.170647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf091205faca492896ba147af407a0c3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.184331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf091205faca492896ba147af407a0c3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.196364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf091205faca492896ba147af407a0c3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.197042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf091205faca492896ba147af407a0c3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.216166 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3390/6707] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:49.219414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.223986 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3318/6708] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:48 2022] DELETE /identity/v3/users/cf091205faca492896ba147af407a0c3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:49.223986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.225082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.226110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.297083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:49.309110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84cd3-ec6e-4151-a551-6c8001b92589 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.317403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6cef2-8d97-4a7c-bd41-a7eedc2010c0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.326198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.326864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.327303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.327531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.328307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.330465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a777c-87d3-4b31-8549-ec1cf17a7b65 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.361867 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3319/6709] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:49.364795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.365027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.365285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.380921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.381645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.381998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.382276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.384332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.388392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:49.389606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.398161 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3320/6710] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:49.399074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.402895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.415245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab86ab-2397-4b7f-aa64-07801d4f8e58 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.421226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.421863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.422145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.422421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] PATH_INFO: `/v3/projects/23e49cba5d5a47148f27d75ccbed2866` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.423666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e49cba5d5a47148f27d75ccbed2866)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.425638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.466943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.468497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.484482 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3391/6711] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:49.485363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.492100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e49cba5d5a47148f27d75ccbed2866', 'request_id': 'req-59c34447-2f39-4a91-8c1c-614394c0fc2f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.500881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232eda3b-c70e-41bc-b5c7-6438dfb6dfd2 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.501864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e49cba5d5a47148f27d75ccbed2866', 'request_id': 'req-59c34447-2f39-4a91-8c1c-614394c0fc2f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.515377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e49cba5d5a47148f27d75ccbed2866', 'request_id': 'req-59c34447-2f39-4a91-8c1c-614394c0fc2f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.541751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] The token cache is being invalidate because project 23e49cba5d5a47148f27d75ccbed2866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:49.542261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.543967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:49.546514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3321/6712] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:49 2022] DELETE /identity/v3/projects/23e49cba5d5a47148f27d75ccbed2866 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:49.546693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.548185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.588289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:49.629516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.630525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.631644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.632438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.634129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.639296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c34447-2f39-4a91-8c1c-614394c0fc2f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.641708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04590b04-391d-491d-8aa9-e47483a61b9e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.643194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.644076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.644362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.644805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.647596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.659963 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3392/6713] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:49.661294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dcddf-7957-48dc-84e9-7e71eb1399d9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.661688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dcddf-7957-48dc-84e9-7e71eb1399d9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.662062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dcddf-7957-48dc-84e9-7e71eb1399d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.676925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.678439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.690225 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3322/6714] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:49.690929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.692415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.706057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03987-c219-4bcd-97c2-c55af9a96c20 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.710928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.711530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.711828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.712076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.712708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.715017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.716203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e82dcddf-7957-48dc-84e9-7e71eb1399d9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:49.717576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e82dcddf-7957-48dc-84e9-7e71eb1399d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.726510 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3323/6715] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:49.730426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.751045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3393/6716] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:49.758522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.758915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.759962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.774406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70503f-edca-4aeb-9ea2-4c413d93ee9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.780939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.781873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.782046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:49.782347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.782754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] PATH_INFO: `/v3/projects/bfa6f980860b4afa85d6fb64a059392b/users/57e10c69312a471d91b109bd881f048d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.783387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.792803 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3394/6717] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:49.797693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.800195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa6f980860b4afa85d6fb64a059392b, user_id=57e10c69312a471d91b109bd881f048d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.803239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a63b12-778c-44d3-a312-4c363617b1db admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.808751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289264d0-8901-43e8-8620-fc6c0b0d112c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.833385 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3324/6718] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:49 2022] PUT /identity/v3/projects/bfa6f980860b4afa85d6fb64a059392b/users/57e10c69312a471d91b109bd881f048d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:49.833591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:49.833591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.833591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.833591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.833904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:49.833904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:49.836134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:49.836268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:49.836426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:49.836805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:49.875481 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3395/6719] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:49.876584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:49.881466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] MFA Rules not processed for user `57e10c69312a471d91b109bd881f048d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:49.882983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:49.899778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e374-5b30-40dd-9b05-87e1d170b175 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:49.917707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:49.993093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3325/6720] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:49.994070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.000776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.001790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.002240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.002890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.003855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.006303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.043932 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3396/6721] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:50.049031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.151562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e4d84-6dcb-4bed-a2e7-1847e85cf6e2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.165404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f415e-27de-4bc3-b1b6-7f897a488ce6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.171358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.172531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.172686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:50.173163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.173533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.174298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.176729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.215797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3397/6722] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:50.220525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.222796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.237180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95963c-7cc7-48cf-a989-ddc4bd5a0544 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.243019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.243802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.244193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.244703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] PATH_INFO: `/v3/projects/b8587c7e1fb24dc684bf5faa899f0761` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.246323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8587c7e1fb24dc684bf5faa899f0761)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.248521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.258669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.259279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.259556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.261689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.262378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.264400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.310941 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3326/6723] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:49 2022] POST /identity/v3/projects => generated 375 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:50.311979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.329278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8587c7e1fb24dc684bf5faa899f0761', 'request_id': 'req-9ad8eae5-0a29-476d-a6ef-87281762e371'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:50.335586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8587c7e1fb24dc684bf5faa899f0761', 'request_id': 'req-9ad8eae5-0a29-476d-a6ef-87281762e371'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:50.348975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8587c7e1fb24dc684bf5faa899f0761', 'request_id': 'req-9ad8eae5-0a29-476d-a6ef-87281762e371'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:50.371684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] The token cache is being invalidate because project b8587c7e1fb24dc684bf5faa899f0761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:50.371684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:50.374410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:50.374410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3398/6724] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:50 2022] DELETE /identity/v3/projects/b8587c7e1fb24dc684bf5faa899f0761 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:50.375260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.376612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.389983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:50.425478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51ed17-ebeb-4cd4-8a07-d3bfeda3c605 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.460950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8eae5-0a29-476d-a6ef-87281762e371 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.468106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.468686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.468960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.469225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.469860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.471985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9c27-e115-4852-95ee-0fd54fedb397 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.499349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.500277 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3399/6725] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:50.500982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.501416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.501540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.501825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.502097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.502197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.503142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.504954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.518163 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3327/6726] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:50.522369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.523274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:50.524601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.534057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3400/6727] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:50.534958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.535120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c78307-913b-44d4-ae6b-dc813a81e3ce admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.536217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.543976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.544665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.545047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.545320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.546223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.548030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f043e0-14c2-49ab-a074-f2ae47b70fa8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.550197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca2613-43d6-4d89-9ccc-0284707cc67d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.555039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.555685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.556105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.556378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.557081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.559765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.562711 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3328/6728] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:50.562711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.562711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.562711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.576835 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3401/6729] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:50.576835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.589271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:50.589554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e230045-dff7-49a1-a059-a431f491a033 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.591357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.594585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.595186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.595506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.595785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.596422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.598126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.602783 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3329/6730] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:50.602783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.611632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957df7ce-eff6-4a87-8bff-cec984bc4028 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.615725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3402/6731] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:50.615725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.621895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.622547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.623506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.623849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] PATH_INFO: `/v3/projects/2ac3d8dc242049bdb81c5d2c5c6e7ca9/users/bc56ce267a8c4c5b97ad466302ef098c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.627026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dfc1c-3b0e-478d-9c61-a40242da38d4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.632748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.633357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.633652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.633908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.634578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:50.634879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.637501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.654118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac3d8dc242049bdb81c5d2c5c6e7ca9, user_id=bc56ce267a8c4c5b97ad466302ef098c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.657068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.658141 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3403/6732] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:14:50.659189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.673414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716ebaf-2797-4966-b93a-db322aa039a8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.687149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3330/6733] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:50 2022] PUT /identity/v3/projects/2ac3d8dc242049bdb81c5d2c5c6e7ca9/users/bc56ce267a8c4c5b97ad466302ef098c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:50.687149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:50.723113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:50.768210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.769155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.769580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.770265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] PATH_INFO: `/v3/projects/637b2069bae94c358cfb2b8b74b1dcb9/users/f1158d92282d4e9ab4519c74a3fab87f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.801304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637b2069bae94c358cfb2b8b74b1dcb9, user_id=f1158d92282d4e9ab4519c74a3fab87f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.804988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bced5-ba7f-47ca-8e47-603932c596a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.824786 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3404/6734] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:50 2022] PUT /identity/v3/projects/637b2069bae94c358cfb2b8b74b1dcb9/users/f1158d92282d4e9ab4519c74a3fab87f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:50.826752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52afac-fbb2-41bf-93dc-02af2a265b61 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.827229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52afac-fbb2-41bf-93dc-02af2a265b61 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.827861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52afac-fbb2-41bf-93dc-02af2a265b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.870553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3154c7-533e-472e-aeba-365fa2c44144 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:50.872115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ea52afac-fbb2-41bf-93dc-02af2a265b61 None None] MFA Rules not processed for user `bc56ce267a8c4c5b97ad466302ef098c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:50.873634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea52afac-fbb2-41bf-93dc-02af2a265b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:50.920648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:50.960685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:50.961295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.961575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.961848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:50.962549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:50.965245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:50.981823 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3405/6735] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:50.986944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:50.987272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:50.987550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.008829 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3331/6736] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/projects => generated 373 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.009970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.039723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] MFA Rules not processed for user `f1158d92282d4e9ab4519c74a3fab87f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:51.041155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:51.123915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678db11-3e3f-453c-8e83-2840c64ddecd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.130548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3406/6737] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:51.130548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.135410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.136469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.136790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.137064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.137708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.140262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.148092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c54e91-b8b5-43de-9c68-1d63fa1f7135 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.156399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.157295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.157755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.158284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.159440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.161722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.179655 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3332/6738] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:51 2022] POST /identity/v3/projects => generated 361 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.180760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.194710 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3407/6739] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:51 2022] POST /identity/v3/users => generated 287 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.195612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.210330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2202b-09ae-4fac-93fe-64b381b2712d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.221246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:51.303093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6ed9b-36b0-4285-a0ea-5420fb9fc217 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.309069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.309704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.309985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.310265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.310950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.313661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.320530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.321407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.321880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.322324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.322984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.326055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.356693 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3333/6740] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.356693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.360825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3408/6741] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:51 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.364611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.376529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c89a9-030a-4188-84fa-b6a4f647a11a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.425053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:51.474921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.475463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.475740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.476008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.476733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.479065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.489359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f49b0-6979-4af5-a7d6-1638cb65c74e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.497590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.498401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.498841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.499210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.500385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.502110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.509065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3409/6742] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:51 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:51.510123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.523689 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3334/6743] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.523689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.525008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3160-23e1-4711-b704-ce9e6601e358 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.541435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.542448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.543029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.544121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.545108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.546656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.548829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824425-f3d2-4b74-bc8f-1f95a882efda admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.557175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.559019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.559480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.560052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.561670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.563915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.566145 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3410/6744] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.567516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.582844 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3335/6745] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.584633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65290a30-4648-4961-9105-3f3f919fb01f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.585382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.593464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.594279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.594629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.594923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.596190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.597371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.618194 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3411/6746] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.619140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.619963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab179-4490-4138-922b-4b8a81ac02c3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.626745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.628500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.629020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.629637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] PATH_INFO: `/v3/projects/8500000b9994466ca9c1d0dabd78d98c/users/1e0db79abb9d45d19229781e87fde2f6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.664998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8500000b9994466ca9c1d0dabd78d98c, user_id=1e0db79abb9d45d19229781e87fde2f6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.665807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8e673-f390-47b4-bbb5-d234961dfcdc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.668809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.674538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.675303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.675821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.676149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] PATH_INFO: `/v3/projects/13a95f862f2e4418a96b06971a1ac113/users/44799587c241463eb17ebf82b2972c1a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.690739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3336/6747] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:51 2022] PUT /identity/v3/projects/8500000b9994466ca9c1d0dabd78d98c/users/1e0db79abb9d45d19229781e87fde2f6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:51.691755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.695151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a95f862f2e4418a96b06971a1ac113, user_id=44799587c241463eb17ebf82b2972c1a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.698424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.718326 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3412/6748] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:51 2022] PUT /identity/v3/projects/13a95f862f2e4418a96b06971a1ac113/users/44799587c241463eb17ebf82b2972c1a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:51.723110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.781543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca1a86-7b5b-45a1-be4b-0b63cd3e634a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.809712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b583-4eb1-47f2-8873-73e1810be7f9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.816007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.816908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.817023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:51.817147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.817527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.818305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.820047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.835028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3413/6749] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.835028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.846047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b8025-6a1b-4a00-9fdc-180ae026ec0d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.851161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.851858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.852203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.852545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.853572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.856445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.869145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.870890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.871627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.871912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] PATH_INFO: `/v3/projects/5a39760733e946d1a73e8d29b389ebb1/users/1e3e073ba18f47d6ad7dab649fa0d0f1/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.890661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a39760733e946d1a73e8d29b389ebb1, user_id=1e3e073ba18f47d6ad7dab649fa0d0f1, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.894475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.898681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3414/6750] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:51.899807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.911399 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3337/6751] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:51 2022] PUT /identity/v3/projects/5a39760733e946d1a73e8d29b389ebb1/users/1e3e073ba18f47d6ad7dab649fa0d0f1/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:51.912305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800475-d6b2-43b4-8b66-4c35665a5a22 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.912525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:51.914382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:51.985968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.986720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.987001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.987269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] PATH_INFO: `/v3/projects/8500000b9994466ca9c1d0dabd78d98c/users/1e0db79abb9d45d19229781e87fde2f6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.989383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8500000b9994466ca9c1d0dabd78d98c, user_id=1e0db79abb9d45d19229781e87fde2f6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:51.990240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb3431-45e7-4a72-8717-6a6c40f540fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:51.992089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:51.995044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:51.996836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:51.997735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:51.998188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:51.998553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] PATH_INFO: `/v3/users/2876ced64d6344e4a4be30a9b09f88d1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:51.999705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2876ced64d6344e4a4be30a9b09f88d1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.001931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.006242 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3415/6752] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:51 2022] PUT /identity/v3/projects/8500000b9994466ca9c1d0dabd78d98c/users/1e0db79abb9d45d19229781e87fde2f6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:52.007488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.067537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2876ced64d6344e4a4be30a9b09f88d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.068209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2876ced64d6344e4a4be30a9b09f88d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.078111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2876ced64d6344e4a4be30a9b09f88d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.086077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2876ced64d6344e4a4be30a9b09f88d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.087162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9252ad9f-37de-479f-8ccb-4e9f74c25da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2876ced64d6344e4a4be30a9b09f88d1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.089709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf8d6e-4253-4530-9639-b1b4b65e751d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.095679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.096741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.097241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.097740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] PATH_INFO: `/v3/projects/13a95f862f2e4418a96b06971a1ac113/users/44799587c241463eb17ebf82b2972c1a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.100720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a95f862f2e4418a96b06971a1ac113, user_id=44799587c241463eb17ebf82b2972c1a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.103748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88db709-f766-4da3-94ba-016502e258a1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.109144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3338/6753] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:51 2022] DELETE /identity/v3/users/2876ced64d6344e4a4be30a9b09f88d1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:52.110670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5a50-c811-4134-b0bf-f1e60d5cea4c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.111084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5a50-c811-4134-b0bf-f1e60d5cea4c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.111370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5a50-c811-4134-b0bf-f1e60d5cea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.116482 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3416/6754] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:52 2022] PUT /identity/v3/projects/13a95f862f2e4418a96b06971a1ac113/users/44799587c241463eb17ebf82b2972c1a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:52.119815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.120141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.120448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.157657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-57cc5a50-c811-4134-b0bf-f1e60d5cea4c None None] MFA Rules not processed for user `1e3e073ba18f47d6ad7dab649fa0d0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:52.158819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57cc5a50-c811-4134-b0bf-f1e60d5cea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.167534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] MFA Rules not processed for user `1e0db79abb9d45d19229781e87fde2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:52.171902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:52.172605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.248407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3339/6755] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:52.248407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.248407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.248407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.280037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] MFA Rules not processed for user `44799587c241463eb17ebf82b2972c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:52.281366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.304139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3417/6756] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:52.304139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.304139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.394265 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3340/6757] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:52.400050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.403378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.406060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fc543-2865-4c2d-9d31-08c15cfe1792 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.411855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.412421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.412705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.412980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.413618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.415804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.446401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:52.460087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3418/6758] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:52 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:52.460087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.460087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.490903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f284a5-db62-4626-b7ee-b2c2ca0c4176 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.540878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31e00a-ec3b-402a-8c5c-8b51ac6073b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.545873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.546608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.546952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.547190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] PATH_INFO: `/v3/projects/c3d65fb083bf46fbaa1ac97228b5b4a2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.548366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d65fb083bf46fbaa1ac97228b5b4a2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.548546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.549176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.549528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.549874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.550280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.556200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:52.607443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d65fb083bf46fbaa1ac97228b5b4a2', 'request_id': 'req-c5bc1440-5538-494b-bcb4-16273a74ae5b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.613236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d65fb083bf46fbaa1ac97228b5b4a2', 'request_id': 'req-c5bc1440-5538-494b-bcb4-16273a74ae5b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.621469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d65fb083bf46fbaa1ac97228b5b4a2', 'request_id': 'req-c5bc1440-5538-494b-bcb4-16273a74ae5b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.633396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.635199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.637505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] The token cache is being invalidate because project c3d65fb083bf46fbaa1ac97228b5b4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:52.637859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.638683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.640502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3419/6759] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:52 2022] DELETE /identity/v3/projects/c3d65fb083bf46fbaa1ac97228b5b4a2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:52.641227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.644772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.652349 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3341/6760] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:52.653626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.665543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d6ff1-605a-4adb-a206-ae3a645373b3 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.740319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc1440-5538-494b-bcb4-16273a74ae5b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.743611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.744297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.745332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.745742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.748640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.756293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.756904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.757185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.757465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.758149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.760477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.791468 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3342/6761] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:52 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:52.792388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.796157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.804496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:52.809158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239d8e-10f4-4746-a219-accd2625fbfb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.815234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.816151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.816841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.817304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] PATH_INFO: `/v3/users/d886993aa3334f53a8e3c2b1a5a00920` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.818676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d886993aa3334f53a8e3c2b1a5a00920)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.821157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.831227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.832973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:52.845061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3420/6762] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:52.846259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.850144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.877284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226994-cc70-4c45-8e08-732503056106 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:52.881733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:52.882345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:52.882658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:52.882911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:52.885646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.886370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd886993aa3334f53a8e3c2b1a5a00920'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.887385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd886993aa3334f53a8e3c2b1a5a00920'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.897655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd886993aa3334f53a8e3c2b1a5a00920'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.911279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd886993aa3334f53a8e3c2b1a5a00920'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.912072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd886993aa3334f53a8e3c2b1a5a00920'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:52.937046 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3343/6763] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:52 2022] DELETE /identity/v3/users/d886993aa3334f53a8e3c2b1a5a00920 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:52.938174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:52.939931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:52.950080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:52.984460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:52.988582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.012155 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3421/6764] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:53.123647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aa6d8-17d6-4810-9ff9-d897281bb864 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.142048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.142851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.143260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.143539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.144334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:53.146140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.175827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3344/6765] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:53.183936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:53.186815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:53.253831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64cae4-1473-424b-a3fc-1e9bfbad245d service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.286419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.287137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.287420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.287701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] PATH_INFO: `/v3/projects/bb49a561d92b4031959716709c0e795e/users/13798d1734dc4d31a4fae7f12035a97c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.325266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb49a561d92b4031959716709c0e795e, user_id=13798d1734dc4d31a4fae7f12035a97c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:53.333013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.360107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3422/6766] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:53 2022] PUT /identity/v3/projects/bb49a561d92b4031959716709c0e795e/users/13798d1734dc4d31a4fae7f12035a97c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:53.362277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:53.382628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349c142-9795-4aa2-a4ef-ca246382d6e4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.514241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:53.515090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:53.554461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.555200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.555561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.555904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.556743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:53.559249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.585060 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3345/6767] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:14:53.590370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:53.602698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566caa6d-72b4-4617-a24a-23685ebb1c68 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.609800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.610581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.610885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.611526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] PATH_INFO: `/v3/projects/bb49a561d92b4031959716709c0e795e/users/13798d1734dc4d31a4fae7f12035a97c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.614100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb49a561d92b4031959716709c0e795e, user_id=13798d1734dc4d31a4fae7f12035a97c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:53.616817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d1ad5-9a1e-4ff1-b8e8-d40d57049a5b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.637567 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3346/6768] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:53 2022] PUT /identity/v3/projects/bb49a561d92b4031959716709c0e795e/users/13798d1734dc4d31a4fae7f12035a97c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:53.642785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc20ac-a2c9-49a9-aa01-35ab61e99e77 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.647295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.647583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.647862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.684786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] MFA Rules not processed for user `13798d1734dc4d31a4fae7f12035a97c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:53.686201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:53.723775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.724761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.725481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.725762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] PATH_INFO: `/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.727030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba5880f95e643b1bbad5e173dcf49c7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:53.730767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:53.741154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:53.781161 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3347/6769] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:53.788245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba5880f95e643b1bbad5e173dcf49c7', 'request_id': 'req-efd29b04-695a-4393-8341-fc5e8e286a25'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:53.789843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:53.791429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:53.796927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba5880f95e643b1bbad5e173dcf49c7', 'request_id': 'req-efd29b04-695a-4393-8341-fc5e8e286a25'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:53.803313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:53.804672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba5880f95e643b1bbad5e173dcf49c7', 'request_id': 'req-efd29b04-695a-4393-8341-fc5e8e286a25'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:53.822571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] The token cache is being invalidate because project 2ba5880f95e643b1bbad5e173dcf49c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:53.825324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:53.826070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:53.842526 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3423/6770] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:53 2022] DELETE /identity/v3/projects/2ba5880f95e643b1bbad5e173dcf49c7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:53.863214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43ad1e-13f6-43de-93df-6283096b3337 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:53.924748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:53.927361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:53.927361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:53.927361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:53.929423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:54.008819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:54.011050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:54.023893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3348/6771] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:54.539405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:54.540863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:54.663206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd29b04-695a-4393-8341-fc5e8e286a25 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:54.685034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:54.685254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:54.685600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:54.686133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] PATH_INFO: `/v3/users/1e3e073ba18f47d6ad7dab649fa0d0f1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:54.688491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3e073ba18f47d6ad7dab649fa0d0f1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:54.691872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:54.773689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3e073ba18f47d6ad7dab649fa0d0f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:54.775405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3e073ba18f47d6ad7dab649fa0d0f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:54.792698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3e073ba18f47d6ad7dab649fa0d0f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:54.825013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3e073ba18f47d6ad7dab649fa0d0f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:54.827295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f731d995-625a-4603-8062-77bec9d25440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3e073ba18f47d6ad7dab649fa0d0f1'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:54.837136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:54.839751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:54.847881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3424/6772] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:54 2022] DELETE /identity/v3/users/1e3e073ba18f47d6ad7dab649fa0d0f1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:54.849503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:54.849962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:54.850586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:54.861881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e15e5-21ef-4d3c-b906-b770b1743771 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:54.875983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:54.877897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:54.889398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:54.949023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:54.950001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:54.950285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:54.950594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] PATH_INFO: `/v3/users/bc56ce267a8c4c5b97ad466302ef098c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:54.952917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc56ce267a8c4c5b97ad466302ef098c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:54.955211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:54.962099 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3425/6773] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:54.974863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:54.976406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:54.989406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:55.056797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc56ce267a8c4c5b97ad466302ef098c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.057787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc56ce267a8c4c5b97ad466302ef098c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.072494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc56ce267a8c4c5b97ad466302ef098c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.076790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d98d2d-6ef5-4d47-98c9-a6b884b4653e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:55.083422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:55.084060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:55.084340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:55.084612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:55.086051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc56ce267a8c4c5b97ad466302ef098c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.087239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-754a20b1-ded7-4473-96e6-36671ac3e47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc56ce267a8c4c5b97ad466302ef098c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.090842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:55.130202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:55.132169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:55.134437 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3349/6774] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:54 2022] DELETE /identity/v3/users/bc56ce267a8c4c5b97ad466302ef098c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:55.139596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:55.140455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:55.141033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:55.148644 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3426/6775] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:55.172353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:55.173572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:55.245259 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3350/6776] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:55.254802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:55.256064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:55.274407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635b63c-5f1e-46ab-b86e-e053e12db579 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:55.344209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:55.344830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:55.345110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:55.345384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:55.364734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:55.366570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:55.411236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3427/6777] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:55.512103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:55.514012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:55.535951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976ca9-e644-4e86-9cca-09c5c2e6a619 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:55.549860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:55.550458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:55.550766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:55.552018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] PATH_INFO: `/v3/projects/5a39760733e946d1a73e8d29b389ebb1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:55.553260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a39760733e946d1a73e8d29b389ebb1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:55.555869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:55.614764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a39760733e946d1a73e8d29b389ebb1', 'request_id': 'req-66ebe551-0087-4d8d-abb8-6e58a8373c1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.622463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a39760733e946d1a73e8d29b389ebb1', 'request_id': 'req-66ebe551-0087-4d8d-abb8-6e58a8373c1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.640287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a39760733e946d1a73e8d29b389ebb1', 'request_id': 'req-66ebe551-0087-4d8d-abb8-6e58a8373c1e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.652905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] The token cache is being invalidate because project 5a39760733e946d1a73e8d29b389ebb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:55.653203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.654015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.655420 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3351/6778] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:55 2022] DELETE /identity/v3/projects/5a39760733e946d1a73e8d29b389ebb1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:55.659397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:55.669757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6692-d88f-4338-b1eb-c63cf550fbe9 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:55.739741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:55.740386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:55.740662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:55.740935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] PATH_INFO: `/v3/users/13798d1734dc4d31a4fae7f12035a97c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:55.742303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13798d1734dc4d31a4fae7f12035a97c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:55.744208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:55.746152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:55.810466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13798d1734dc4d31a4fae7f12035a97c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.811206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13798d1734dc4d31a4fae7f12035a97c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.819920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13798d1734dc4d31a4fae7f12035a97c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.833678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13798d1734dc4d31a4fae7f12035a97c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.834488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13798d1734dc4d31a4fae7f12035a97c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:55.850549 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3428/6779] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:55 2022] DELETE /identity/v3/users/13798d1734dc4d31a4fae7f12035a97c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:55.912826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:55.914323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:55.930615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebe551-0087-4d8d-abb8-6e58a8373c1e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:55.947832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:56.003077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.003972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.004286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.005368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] PATH_INFO: `/v3/projects/2ac3d8dc242049bdb81c5d2c5c6e7ca9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.007178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac3d8dc242049bdb81c5d2c5c6e7ca9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.009700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.076082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac3d8dc242049bdb81c5d2c5c6e7ca9', 'request_id': 'req-996a33b1-2596-41de-88bd-87c7c7fc1bf1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.083474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac3d8dc242049bdb81c5d2c5c6e7ca9', 'request_id': 'req-996a33b1-2596-41de-88bd-87c7c7fc1bf1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.093548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac3d8dc242049bdb81c5d2c5c6e7ca9', 'request_id': 'req-996a33b1-2596-41de-88bd-87c7c7fc1bf1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.108050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] The token cache is being invalidate because project 2ac3d8dc242049bdb81c5d2c5c6e7ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:56.108265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.108832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.110602 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3352/6780] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:55 2022] DELETE /identity/v3/projects/2ac3d8dc242049bdb81c5d2c5c6e7ca9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:56.114541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.116429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.130713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd6e25-ff6d-4ca5-ac3c-1be34b52a94b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.202355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.205990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.220935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.221613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.221921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.222198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] PATH_INFO: `/v3/users/1e0db79abb9d45d19229781e87fde2f6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.223773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0db79abb9d45d19229781e87fde2f6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.226933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a33b1-2596-41de-88bd-87c7c7fc1bf1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.229309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.235424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.236768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.237936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.238262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] PATH_INFO: `/v3/projects/bb49a561d92b4031959716709c0e795e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.239716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb49a561d92b4031959716709c0e795e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.242058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.317293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb49a561d92b4031959716709c0e795e', 'request_id': 'req-0246063a-0cdb-4828-87f9-ead35da03cf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.328930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb49a561d92b4031959716709c0e795e', 'request_id': 'req-0246063a-0cdb-4828-87f9-ead35da03cf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.333372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0db79abb9d45d19229781e87fde2f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.334311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0db79abb9d45d19229781e87fde2f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.343977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb49a561d92b4031959716709c0e795e', 'request_id': 'req-0246063a-0cdb-4828-87f9-ead35da03cf5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.347977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0db79abb9d45d19229781e87fde2f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.366319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] The token cache is being invalidate because project bb49a561d92b4031959716709c0e795e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:56.366796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.367254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0db79abb9d45d19229781e87fde2f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.367534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0246063a-0cdb-4828-87f9-ead35da03cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.368054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4ef306fa-d61a-4a6c-8396-7ea3c612c841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0db79abb9d45d19229781e87fde2f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.369710 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3353/6781] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:56 2022] DELETE /identity/v3/projects/bb49a561d92b4031959716709c0e795e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:56.376296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.376595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.376876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.383183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:56.393209 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3429/6782] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:56 2022] DELETE /identity/v3/users/1e0db79abb9d45d19229781e87fde2f6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:56.420104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:56.421886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.460306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:56.551860 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3354/6783] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:56.553305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.553590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.553866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.581885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.583345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:56.583470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.584765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.593603 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3430/6784] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:56.597963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.599874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385676ed-b2b8-460d-bc7d-4f175432a8f4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.606294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.606986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.607269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.607544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] PATH_INFO: `/v3/users/57e10c69312a471d91b109bd881f048d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.608748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e3ae1-ef94-4bb5-900d-a24864adf06a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.608875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e10c69312a471d91b109bd881f048d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.610888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.613884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.614902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.615285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.615650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.616399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:56.617403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.618550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.655476 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3431/6785] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:14:56.662809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.676269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e898-210e-40d8-91a4-556e4cdcc091 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.684949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.685650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.685946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.686219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.687056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.689085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.696889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e10c69312a471d91b109bd881f048d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.698188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e10c69312a471d91b109bd881f048d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.709003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e10c69312a471d91b109bd881f048d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.721872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e10c69312a471d91b109bd881f048d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.723353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-88f267da-a245-4da1-b75e-cafbab906d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e10c69312a471d91b109bd881f048d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.729944 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3432/6786] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:56 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:56.731114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.733234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.756185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3355/6787] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:56 2022] DELETE /identity/v3/users/57e10c69312a471d91b109bd881f048d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:56.758924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.759496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.759786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.787449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:56.788941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.827890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1ee3-114a-4736-a3a1-f19d771cb92f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.872743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3356/6788] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:56.872743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.874792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.890165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e4df4-6c1e-4a2b-a4aa-b5197bbeabd9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:56.896594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.897455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.897894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.898183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] PATH_INFO: `/v3/projects/8500000b9994466ca9c1d0dabd78d98c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.899752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8500000b9994466ca9c1d0dabd78d98c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.901757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.913072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:56.915766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:56.915766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:56.915766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:56.915766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:56.917589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:56.945583 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3433/6789] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:56 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:56.946384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:56.948934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:56.955811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:56.968058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8500000b9994466ca9c1d0dabd78d98c', 'request_id': 'req-fc8c4b1b-debc-4751-9bee-8dd1d541938b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.980880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8500000b9994466ca9c1d0dabd78d98c', 'request_id': 'req-fc8c4b1b-debc-4751-9bee-8dd1d541938b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:56.991938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8500000b9994466ca9c1d0dabd78d98c', 'request_id': 'req-fc8c4b1b-debc-4751-9bee-8dd1d541938b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.007844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] The token cache is being invalidate because project 8500000b9994466ca9c1d0dabd78d98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:57.008262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.008843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.015885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3357/6790] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:56 2022] DELETE /identity/v3/projects/8500000b9994466ca9c1d0dabd78d98c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:57.015885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.015885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.040343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3954e8-2460-4265-be76-c858f909e25c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.046062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:57.094078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4b1b-debc-4751-9bee-8dd1d541938b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.099663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.100380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.100660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.101002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.101673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.102644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.103361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.103481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2790e55-1b05-4fc0-89ad-84469ba65cd2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.103663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.103913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.108914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.119023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3358/6791] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:57.119023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3091b-0d0d-4445-b2ad-367392d8eff7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.119023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3091b-0d0d-4445-b2ad-367392d8eff7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.119023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3091b-0d0d-4445-b2ad-367392d8eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.122752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.124206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.135167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3434/6792] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:57.136816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.147269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdfb759-205a-4db0-8edb-418d64459374 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.149571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-18f3091b-0d0d-4445-b2ad-367392d8eff7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:57.151698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-18f3091b-0d0d-4445-b2ad-367392d8eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.152889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.153701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.154053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.154397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.155138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.157132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.163197 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3359/6793] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:57.170586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3435/6794] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:57.172618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.172618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.172853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.174187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.194534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732d8e-83a6-4cd1-a2cc-f9ec616c0ceb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.196892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:57.198947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.204386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.205180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.206181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.206181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] PATH_INFO: `/v3/projects/fd7fe4052d0741a3898dbfce6bc35b95/users/8277766423324152ab482de27bf1a149/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.210643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3360/6795] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:57.214132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.224694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da960736-38ec-45bb-9e87-6153eb667070 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.231455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.232328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.232328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.232644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.233334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:57.233683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.236642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.250097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7fe4052d0741a3898dbfce6bc35b95, user_id=8277766423324152ab482de27bf1a149, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.252717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720069-9548-4b94-b7d7-569a42654e30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.254775 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3361/6796] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:14:57.255678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.269458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779f17e-5828-49af-b40c-a6c94d0cba0b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.282336 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3436/6797] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:57 2022] PUT /identity/v3/projects/fd7fe4052d0741a3898dbfce6bc35b95/users/8277766423324152ab482de27bf1a149/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:57.284133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.284550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.285920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.332019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] MFA Rules not processed for user `8277766423324152ab482de27bf1a149`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:57.333435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.374785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.376176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.376646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.377550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.378364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.383443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.442488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3362/6798] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:57.447043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.476652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:57.484665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3437/6799] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:57.488878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.509043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63d41c-1629-47d4-a7ea-dbafbc39e2d9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.582922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d5dae-45e0-47ea-8156-9d6724b715bf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.594627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.595188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.595323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.595637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.596336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.600037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.639130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.640145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.640275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.640607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.641496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.643902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.654703 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3363/6800] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:57.655855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.657945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.658347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:57.676785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e454d2-4c60-494d-93b8-bbc69909c2b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.685754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.686840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.687320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.688708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] PATH_INFO: `/v3/projects/bfa6f980860b4afa85d6fb64a059392b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.690696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa6f980860b4afa85d6fb64a059392b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.695782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.717747 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3438/6801] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:57.719744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.766398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa6f980860b4afa85d6fb64a059392b', 'request_id': 'req-79f7b8b0-00d9-411a-acab-6ea76c7487bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.774003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa6f980860b4afa85d6fb64a059392b', 'request_id': 'req-79f7b8b0-00d9-411a-acab-6ea76c7487bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.785337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa6f980860b4afa85d6fb64a059392b', 'request_id': 'req-79f7b8b0-00d9-411a-acab-6ea76c7487bd'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.805122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] The token cache is being invalidate because project bfa6f980860b4afa85d6fb64a059392b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:57.805414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.806070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:57.808818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3364/6802] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:57 2022] DELETE /identity/v3/projects/bfa6f980860b4afa85d6fb64a059392b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:57.811265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.812726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.815186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fa031-bb7e-4603-b4e4-6d783da7940a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.882722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.883337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.883761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.884147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.884788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.886627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.891048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7b8b0-00d9-411a-acab-6ea76c7487bd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.896788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.897417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.898040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.898305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.898927 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3439/6803] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:57.899052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.899782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.901884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.903167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.916433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc05cf-f1a7-47eb-a8ad-28fd0ec6711b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.923474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.924141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.924416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.924783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] PATH_INFO: `/v3/users/f1158d92282d4e9ab4519c74a3fab87f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.925940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1158d92282d4e9ab4519c74a3fab87f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.927845 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3365/6804] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:57 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:57.927976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.929355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.934975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:57.959068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d027-3b5e-4d2d-84d5-8aa384c0d3e9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:57.964953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:57.967291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:57.967401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:57.967941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:57.968916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:57.970723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:57.987551 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3366/6805] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:57.988562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:57.999759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65131fdf-2e74-46a7-9ab9-c0ba69129d93 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.004508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.005089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.005370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.005640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.006263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.009362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.011381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1158d92282d4e9ab4519c74a3fab87f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.012518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1158d92282d4e9ab4519c74a3fab87f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.028219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1158d92282d4e9ab4519c74a3fab87f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.030049 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3367/6806] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:58.034225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.037622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.050385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:58.051380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1158d92282d4e9ab4519c74a3fab87f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.052873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1158d92282d4e9ab4519c74a3fab87f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.058963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825ff04-115c-49c3-aa0a-bad364f3f2f7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.066198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.067008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.067347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.067629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] PATH_INFO: `/v3/projects/bcdcaa646ba54f75abea17bdd8a6a62d/users/82a854bf2b0042439e87b8aa0476b0e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.074074 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3440/6807] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:57 2022] DELETE /identity/v3/users/f1158d92282d4e9ab4519c74a3fab87f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:58.074074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.075415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.086149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdcaa646ba54f75abea17bdd8a6a62d, user_id=82a854bf2b0042439e87b8aa0476b0e4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.090377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b2c67-469d-4f9b-b563-40a832f7f222 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.092452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:58.111758 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3368/6808] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:58 2022] PUT /identity/v3/projects/bcdcaa646ba54f75abea17bdd8a6a62d/users/82a854bf2b0042439e87b8aa0476b0e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:58.117142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.117544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.117900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.148294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:58.149520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.179899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf3230-a9d6-464e-ad6e-a51e361b93c5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.271590 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3369/6809] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:58.273225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.273912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.274190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.274464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] PATH_INFO: `/v3/projects/450936eefeae4619b5f27a147973c048/users/8f78850d2b474c6d87c50378713c66c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.283002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.283002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.315644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=450936eefeae4619b5f27a147973c048, user_id=8f78850d2b474c6d87c50378713c66c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.318536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.345269 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3441/6810] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:58 2022] PUT /identity/v3/projects/450936eefeae4619b5f27a147973c048/users/8f78850d2b474c6d87c50378713c66c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:58.349097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.386022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801c333-169e-4de2-87a2-9b0ab39b6f97 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.474338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.475778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.476262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.476841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.481822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f691e2-38c7-4251-92cd-a974fdbe2b98 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.492393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.493530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.494446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.495426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.495805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.496483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.496644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.498583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.509518 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3370/6811] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:14:58.512414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.514136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.514252 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3442/6812] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:58.515095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.532460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979f3ea-34b9-436b-8c27-78ba15c6d275 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.534806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88384a-df33-417b-849c-61fff940330c service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.537476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.538220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.538684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.539227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] PATH_INFO: `/v3/projects/450936eefeae4619b5f27a147973c048/users/8f78850d2b474c6d87c50378713c66c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.541949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=450936eefeae4619b5f27a147973c048, user_id=8f78850d2b474c6d87c50378713c66c1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.542345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.543209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.543494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.543767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] PATH_INFO: `/v3/projects/637b2069bae94c358cfb2b8b74b1dcb9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.544922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] RBAC: Authorizing `identity:delete_project(project_id=637b2069bae94c358cfb2b8b74b1dcb9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.546057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3978b-d04b-457d-825a-7c0012406945 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.546915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.570165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3443/6813] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:58 2022] PUT /identity/v3/projects/450936eefeae4619b5f27a147973c048/users/8f78850d2b474c6d87c50378713c66c1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:58.572293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.572724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.573493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.598730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:58.618417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] MFA Rules not processed for user `8f78850d2b474c6d87c50378713c66c1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:58.619869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.626542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637b2069bae94c358cfb2b8b74b1dcb9', 'request_id': 'req-004aed54-88c2-4534-af1d-931396d5000c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.630127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:58.646278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637b2069bae94c358cfb2b8b74b1dcb9', 'request_id': 'req-004aed54-88c2-4534-af1d-931396d5000c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.660138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637b2069bae94c358cfb2b8b74b1dcb9', 'request_id': 'req-004aed54-88c2-4534-af1d-931396d5000c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.684820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] The token cache is being invalidate because project 637b2069bae94c358cfb2b8b74b1dcb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:58.685305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.687465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.691869 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3371/6814] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:58 2022] DELETE /identity/v3/projects/637b2069bae94c358cfb2b8b74b1dcb9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:58.693305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.697035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.749655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3444/6815] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:58.762959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:58.764885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.791807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aed54-88c2-4534-af1d-931396d5000c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.797913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.798533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.799517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.799899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] PATH_INFO: `/v3/users/44799587c241463eb17ebf82b2972c1a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.800990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=44799587c241463eb17ebf82b2972c1a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.802967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.881801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44799587c241463eb17ebf82b2972c1a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.883847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818034f-a1e6-4264-b749-5ef9c80eb161 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:58.884431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44799587c241463eb17ebf82b2972c1a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.892860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:58.893493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:58.893775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:58.894816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:58.898623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:58.904245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44799587c241463eb17ebf82b2972c1a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.917979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44799587c241463eb17ebf82b2972c1a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.919110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44799587c241463eb17ebf82b2972c1a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:58.920050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:58.921662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:58.935970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3445/6816] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:58.967866 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3372/6817] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:14:58 2022] DELETE /identity/v3/users/44799587c241463eb17ebf82b2972c1a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:59.031290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.032842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.062276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152a075-3c68-4b7a-9b5f-5515ed5a55fa service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.133477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.134112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.134504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.134932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.138090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.206969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.208511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9ea4-a7c8-43e7-a309-245aa1c8a0af service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.218457 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3446/6818] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:14:59.441986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.444210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.464011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.464632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.465007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.472996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:59.498837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] MFA Rules not processed for user `82a854bf2b0042439e87b8aa0476b0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:59.499042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:59.500093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.537936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952ce70-fb97-4db9-8630-67798c2266ba admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.543944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.544913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.545266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.545531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] PATH_INFO: `/v3/projects/13a95f862f2e4418a96b06971a1ac113` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.546720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a95f862f2e4418a96b06971a1ac113)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.548669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.589050 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3447/6819] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:59.590843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.592202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.608289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9e078-2b9c-4aac-a8a4-a721c0e78044 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.619002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a95f862f2e4418a96b06971a1ac113', 'request_id': 'req-0575594d-219f-4d8c-a95c-e00d1c152062'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:59.626312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a95f862f2e4418a96b06971a1ac113', 'request_id': 'req-0575594d-219f-4d8c-a95c-e00d1c152062'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:59.641990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a95f862f2e4418a96b06971a1ac113', 'request_id': 'req-0575594d-219f-4d8c-a95c-e00d1c152062'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:59.660435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] The token cache is being invalidate because project 13a95f862f2e4418a96b06971a1ac113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:14:59.660674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:59.661745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0575594d-219f-4d8c-a95c-e00d1c152062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:14:59.667141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3373/6820] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:14:59 2022] DELETE /identity/v3/projects/13a95f862f2e4418a96b06971a1ac113 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:14:59.668727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbbe84-cd03-4d06-bca4-93e2f6ff5246 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.669018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbbe84-cd03-4d06-bca4-93e2f6ff5246 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.669297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbbe84-cd03-4d06-bca4-93e2f6ff5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.692078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f0fbbe84-cd03-4d06-bca4-93e2f6ff5246 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:59.693307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f0fbbe84-cd03-4d06-bca4-93e2f6ff5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.696185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.696958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.697313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.698696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.699459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.701302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.703328 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3374/6821] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:14:59.712254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.712538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.712955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.735189 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3448/6822] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:59.739009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.740946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.743365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:14:59.744846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:14:59.828719 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3375/6823] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:14:59.829405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32f620a-b211-404b-89aa-e35642d056c2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.834125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.835676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.836367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.836770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.837113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.837894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.841552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.845481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac0bd9-29f2-4357-b0e4-edab7d497eb6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.850051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.851063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.851423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.851911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.852575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:14:59.853402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.855995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.867144 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3449/6824] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:14:59.871287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.878556 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3376/6825] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:14:59.879480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.894523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0f35-a437-4573-8705-b901d7bad6cf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.896739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaf5db-4a89-4e92-92d9-62b51507f911 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.903279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.904025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.904374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.904717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.905434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.907285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.909035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:59.919099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.919935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.920407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.920691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.921358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.925871 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3450/6826] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:14:59.926006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.928466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.953586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa62a56-a249-4039-9547-ae4a34c8faca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:14:59.958587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:14:59.959328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:14:59.959462 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3377/6827] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:14:59.959903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:14:59.960184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] PATH_INFO: `/v3/projects/0117cd4ebb924451a1388a45aeac74e6/users/212241b715754f02a85a0e0de7cca15c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:14:59.965488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:14:59.976950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0117cd4ebb924451a1388a45aeac74e6, user_id=212241b715754f02a85a0e0de7cca15c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:14:59.980052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:14:59.994280 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3451/6828] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:14:59 2022] PUT /identity/v3/projects/0117cd4ebb924451a1388a45aeac74e6/users/212241b715754f02a85a0e0de7cca15c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:14:59.997540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:14:59.998411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.032474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f158b79-b23b-4c43-8eaf-b1694416e114 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.063968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159d8f9-78d0-4d91-8f13-c9377f986c0d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.068898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.069507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.069791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.070069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.070753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.072367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.083907 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3452/6829] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:00.087719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.099938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.100552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.100846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.101136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.101387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91aba6-ee01-4d0b-8b1d-cc32ec0c0e42 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.106924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.107586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.107869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.108164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] PATH_INFO: `/v3/projects/0117cd4ebb924451a1388a45aeac74e6/users/212241b715754f02a85a0e0de7cca15c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.110099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0117cd4ebb924451a1388a45aeac74e6, user_id=212241b715754f02a85a0e0de7cca15c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.113447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.113585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.113696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.131240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3453/6830] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:00 2022] PUT /identity/v3/projects/0117cd4ebb924451a1388a45aeac74e6/users/212241b715754f02a85a0e0de7cca15c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:00.144222 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3378/6831] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:14:59 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:00.148006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.222683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b12afd-dbdf-47f5-b551-345264a46749 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.227889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.228464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.228751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.229217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.229884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.231555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.243631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3454/6832] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:00.248328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.262310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66931b0e-0df5-466c-932e-f39e77172fbf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.267810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.268526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.268946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.269337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.270136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.271859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f651e6-ff32-40fe-9880-01bae93e8331 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.285459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3379/6833] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:00.290244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.301819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b703ee-4ad1-4d39-aa41-751d9bd73af8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.308590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.309541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.309892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.310273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] PATH_INFO: `/v3/projects/bff6a52231e946ba9aa9c12a9838d417/users/79c27c5bcbd74356970c974ad356a705/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.335357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff6a52231e946ba9aa9c12a9838d417, user_id=79c27c5bcbd74356970c974ad356a705, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.339295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.362147 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3455/6834] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:00 2022] PUT /identity/v3/projects/bff6a52231e946ba9aa9c12a9838d417/users/79c27c5bcbd74356970c974ad356a705/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:00.364306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.364941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.365368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.397006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:00.398354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:00.476324 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3380/6835] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:00.532160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.534803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:00.578132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:00.649132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e78055-e04c-41ca-8c3a-87152763f7fe admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.658099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.659232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.659653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.662877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.678575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.680848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3000e8-440b-4da9-858d-e85ab0eb8d98 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.700413 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3456/6836] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:00.761370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:00.763596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:00.778862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcd6e1-c9dd-4a52-b4cc-a6e0c94df997 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:00.787391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:00.787391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.787688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.787980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] PATH_INFO: `/v3/users/8277766423324152ab482de27bf1a149` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.791222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8277766423324152ab482de27bf1a149)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:00.793978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:00.864556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8277766423324152ab482de27bf1a149'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:00.865320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8277766423324152ab482de27bf1a149'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:00.868439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:00.876629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8277766423324152ab482de27bf1a149'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:00.888110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8277766423324152ab482de27bf1a149'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:00.888832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8277766423324152ab482de27bf1a149'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:00.931950 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3381/6837] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:00 2022] DELETE /identity/v3/users/8277766423324152ab482de27bf1a149 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:00.933520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49ed10-f791-466f-b69a-6045b19832dc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:00.933996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49ed10-f791-466f-b69a-6045b19832dc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:00.934074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49ed10-f791-466f-b69a-6045b19832dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:00.979687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2d49ed10-f791-466f-b69a-6045b19832dc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:00.981247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2d49ed10-f791-466f-b69a-6045b19832dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.077294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3457/6838] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:01.087182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.089475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.107852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.108266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.108660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.152295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] MFA Rules not processed for user `212241b715754f02a85a0e0de7cca15c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:01.153814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.210757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b015d-eef5-4d0c-9b02-870be40071ee admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:01.223814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:01.224737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.225202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.225664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.247447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:01.249112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151c43f-ffbb-4aec-99a4-b65342fbdbb2 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:01.261372 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3382/6839] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:01.265422 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3458/6840] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:01.277953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.278814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.278814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.300382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] MFA Rules not processed for user `82a854bf2b0042439e87b8aa0476b0e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:01.301346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.380705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3383/6841] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:01.395752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.398442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.462133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:01.487420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6c92-34ee-42c8-942f-2a2d6bcfd058 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:01.493148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:01.493760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.494041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.494320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.496128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.508175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:01.509858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ec7b-dd99-41f1-9c4c-c4fa11e1fff2 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:01.519979 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3459/6842] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:01.555471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.557665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.658683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf60ae-4985-4ab4-817b-c406ebb0a92d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:01.662984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:01.663768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.664105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.664392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.666896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.698867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:01.701316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:01.711038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.711326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.711599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.718994 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3384/6843] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:01.754409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] MFA Rules not processed for user `79c27c5bcbd74356970c974ad356a705`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:01.756020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.821313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.823504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.845272 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3460/6844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:01.851660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90b3c-cfe1-4b1d-8d79-43029913ef13 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:01.855625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.857505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:01.858698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:01.859474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:01.859859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:01.860235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] PATH_INFO: `/v3/projects/fd7fe4052d0741a3898dbfce6bc35b95` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:01.861512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7fe4052d0741a3898dbfce6bc35b95)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:01.863705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:01.867074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:01.936357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7fe4052d0741a3898dbfce6bc35b95', 'request_id': 'req-da5aee3b-6d04-4884-874b-6c854538ccde'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:01.945020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7fe4052d0741a3898dbfce6bc35b95', 'request_id': 'req-da5aee3b-6d04-4884-874b-6c854538ccde'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:01.952864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17121-211c-494d-af1b-5e3bd8740245 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:01.959286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7fe4052d0741a3898dbfce6bc35b95', 'request_id': 'req-da5aee3b-6d04-4884-874b-6c854538ccde'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:01.977627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] The token cache is being invalidate because project fd7fe4052d0741a3898dbfce6bc35b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:01.977838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:01.978671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:01.981779 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3385/6845] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:01 2022] DELETE /identity/v3/projects/fd7fe4052d0741a3898dbfce6bc35b95 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:01.985567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:01.987414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:02.011554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:02.012176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.012459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.012730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.014702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:02.074523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5aee3b-6d04-4884-874b-6c854538ccde admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:02.074523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:02.074523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.074523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.074992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] PATH_INFO: `/v3/users/8f78850d2b474c6d87c50378713c66c1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.074992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f78850d2b474c6d87c50378713c66c1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:02.074992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:02.083865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:02.107716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:02.109546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:02.120058 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3461/6846] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:02.136810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f78850d2b474c6d87c50378713c66c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.137793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f78850d2b474c6d87c50378713c66c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.147832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f78850d2b474c6d87c50378713c66c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.157506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f78850d2b474c6d87c50378713c66c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.158342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a440e994-6fa5-46c9-b32e-93e51af53619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f78850d2b474c6d87c50378713c66c1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.184467 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3386/6847] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:01 2022] DELETE /identity/v3/users/8f78850d2b474c6d87c50378713c66c1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:02.558210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:02.573427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ecd59-b822-4a15-812e-11bd43b07cc2 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:02.640130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:02.640802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.641086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.641354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] PATH_INFO: `/v3/projects/450936eefeae4619b5f27a147973c048` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.642475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450936eefeae4619b5f27a147973c048)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:02.644411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:02.696243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450936eefeae4619b5f27a147973c048', 'request_id': 'req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.702167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450936eefeae4619b5f27a147973c048', 'request_id': 'req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.709006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450936eefeae4619b5f27a147973c048', 'request_id': 'req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.724905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] The token cache is being invalidate because project 450936eefeae4619b5f27a147973c048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:02.725201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.725824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ff8cb0-5a4a-4972-8537-7ea9cad29a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:02.727595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3462/6848] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:02 2022] DELETE /identity/v3/projects/450936eefeae4619b5f27a147973c048 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:02.735111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb666f98-706a-471a-beeb-d929026d708c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.735412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb666f98-706a-471a-beeb-d929026d708c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.735765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb666f98-706a-471a-beeb-d929026d708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.756289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cb666f98-706a-471a-beeb-d929026d708c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:02.757642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb666f98-706a-471a-beeb-d929026d708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:02.763629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:02.827325 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3387/6849] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:02.836300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.836582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.836922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.860123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:02.861357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:02.871297 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3463/6850] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:02.875829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb666f98-706a-471a-beeb-d929026d708c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:02.886294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb666f98-706a-471a-beeb-d929026d708c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:02.891088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:02.891698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.892724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.892949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.893597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:02.893898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:02.895554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:02.908528 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3388/6851] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:15:02.913239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:02.923390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0773e-ab13-4732-9d09-53b67e3f528f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:02.928154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:02.928754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:02.929028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:02.929298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:02.929971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:02.931967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:02.964835 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3464/6852] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:02 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:02.966842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:02.978396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a10cbc-99ee-44f0-8ab0-cd19d57f4304 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.037799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.038553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.038834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.039142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.039776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.042287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.066434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3389/6853] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:02 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:03.070637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.084319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64c448-b02d-4ec9-b3b6-7388d6a7893d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.091232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.091937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.092274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.092643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.093644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.095299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.108836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3465/6854] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:03.113062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.125753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db0cc6-f940-492d-b98d-c97d5778a2a7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.133444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.134437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.135741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.136612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.137538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.139796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ce78c-a0be-4321-ad83-6ee637894e3e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.158438 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3390/6855] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:03.159667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.171729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426632a-72dd-44cd-95de-27bfafabbe84 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.176502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.177168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.177432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.177704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] PATH_INFO: `/v3/projects/1d9cbff9c0c5434eab9f681a4f94de5d/users/110edd64c1d34eeca08390b8573e8d49/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.193585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9cbff9c0c5434eab9f681a4f94de5d, user_id=110edd64c1d34eeca08390b8573e8d49, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.196373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.211213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3466/6856] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:03 2022] PUT /identity/v3/projects/1d9cbff9c0c5434eab9f681a4f94de5d/users/110edd64c1d34eeca08390b8573e8d49/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:03.221292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.221577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.221855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.250457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] MFA Rules not processed for user `110edd64c1d34eeca08390b8573e8d49`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:03.251617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:03.318660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3391/6857] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:03.324443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.333358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:03.389220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abeb14-34fa-4282-828c-79db11217fbb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.393927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.394544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.394838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.395091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.395718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.397597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.425497 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3467/6858] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:03 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:03.429632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.438343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:03.443003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ad590-5930-439a-aebf-86e7ac286e0c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.503881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.504409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.504689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.505483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.505579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.507585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.539597 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3392/6859] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:03 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:03.541086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.555819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8aef8-37a9-432f-a4fb-621825cbe542 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.564674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.565503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.565806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.566076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.566758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.568339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.586318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3468/6860] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:03.587159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.601066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ceee-36cd-43cc-baeb-8c9796a9f2e9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.608469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.609269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.610065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.610353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] PATH_INFO: `/v3/projects/552c3413815d4b558bf99434a5fbecca/users/09befe431f3446b8ae6854e92f629a97/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.633255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552c3413815d4b558bf99434a5fbecca, user_id=09befe431f3446b8ae6854e92f629a97, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.639207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.672358 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3393/6861] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:03 2022] PUT /identity/v3/projects/552c3413815d4b558bf99434a5fbecca/users/09befe431f3446b8ae6854e92f629a97/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:03.674140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.688014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479a355-03c3-43ce-8dbc-f657ab77a96b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.783216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.783934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.784218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.784665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.785327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.787211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1cb7e-8302-4e90-a0c4-267e9e2e17ea admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.802545 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3469/6862] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:03.804901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.817347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c896c-1f8d-4a63-a1f7-22154343a369 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:03.822880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:03.823932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.823932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.824185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] PATH_INFO: `/v3/projects/552c3413815d4b558bf99434a5fbecca/users/09befe431f3446b8ae6854e92f629a97/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.826438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=552c3413815d4b558bf99434a5fbecca, user_id=09befe431f3446b8ae6854e92f629a97, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:03.829918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:03.844215 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3394/6863] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:03 2022] PUT /identity/v3/projects/552c3413815d4b558bf99434a5fbecca/users/09befe431f3446b8ae6854e92f629a97/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:03.854955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:03.855247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:03.855526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:03.886694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] MFA Rules not processed for user `09befe431f3446b8ae6854e92f629a97`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:03.888598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:03.966471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3470/6864] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:03.981806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:03.984670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:04.067056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3f76-1c68-4415-a7b6-b47a626dfcda admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.071623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.072235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.072565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.072848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.084205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.086263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.099812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3395/6865] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:04.508882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:04.515647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:04.521453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd85d1-504b-4a38-b2cf-910a0c8c8885 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.529307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09befe431f3446b8ae6854e92f629a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '552c3413815d4b558bf99434a5fbecca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.529935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.530221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.530491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.531168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:04.531471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.533149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.551284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3471/6866] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:04.552440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:04.562577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf3d3-6b38-4875-9e46-6dc47f2d4bf7 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.568449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09befe431f3446b8ae6854e92f629a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '552c3413815d4b558bf99434a5fbecca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.569723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.569723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.569723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.570396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.573397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.607190 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3396/6867] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:04 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:04.612060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:04.624887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a5119-8f08-4381-940b-99f56f8f69b2 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.665824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:04.669978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:04.707485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09befe431f3446b8ae6854e92f629a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '552c3413815d4b558bf99434a5fbecca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.708165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.708737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.708848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.709449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:04.709756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.711618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.724971 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3472/6868] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:04.725552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:04.755138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fe9e1-b3a7-43a7-a47e-8cee0c58c971 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.763402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.764054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.764336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.764613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] PATH_INFO: `/v3/users/79c27c5bcbd74356970c974ad356a705` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.765895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c27c5bcbd74356970c974ad356a705)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.767960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.788565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:04.802223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d415b-5b6a-43f2-a6b0-e8bb050118d7 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:04.809677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09befe431f3446b8ae6854e92f629a97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '552c3413815d4b558bf99434a5fbecca', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:04.810753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:04.812067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:04.812562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] PATH_INFO: `/v3/projects/a149b7dc8ff0468b913b1ff0ae54033f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:04.814752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a149b7dc8ff0468b913b1ff0ae54033f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:04.817180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:04.834419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c27c5bcbd74356970c974ad356a705'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.835240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c27c5bcbd74356970c974ad356a705'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.847524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c27c5bcbd74356970c974ad356a705'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.862259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c27c5bcbd74356970c974ad356a705'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.864716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c27c5bcbd74356970c974ad356a705'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.889814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a149b7dc8ff0468b913b1ff0ae54033f', 'request_id': 'req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.900475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3397/6869] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:04 2022] DELETE /identity/v3/users/79c27c5bcbd74356970c974ad356a705 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:04.900673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a149b7dc8ff0468b913b1ff0ae54033f', 'request_id': 'req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.907639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a149b7dc8ff0468b913b1ff0ae54033f', 'request_id': 'req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.925317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] The token cache is being invalidate because project a149b7dc8ff0468b913b1ff0ae54033f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:04.925658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.927018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:04.928914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3473/6870] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:04 2022] DELETE /identity/v3/projects/a149b7dc8ff0468b913b1ff0ae54033f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:05.169625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.171269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.244982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.247577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.257490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242df551-d771-41da-86d4-6e8f55610e49 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.263811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.265918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.265918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.265918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] PATH_INFO: `/v3/users/110edd64c1d34eeca08390b8573e8d49` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.267117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110edd64c1d34eeca08390b8573e8d49)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.269172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.363150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110edd64c1d34eeca08390b8573e8d49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.363937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110edd64c1d34eeca08390b8573e8d49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.374390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110edd64c1d34eeca08390b8573e8d49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.379775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172f9b-c749-4d3f-b94c-f1e5dcafdfd3 tempest-VolumeQuotaClassesTest-640591021 tempest-VolumeQuotaClassesTest-640591021-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.381781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:05.385514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.386248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.386653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.386933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] PATH_INFO: `/v3/projects/bff6a52231e946ba9aa9c12a9838d417` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.387432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110edd64c1d34eeca08390b8573e8d49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.388182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff6a52231e946ba9aa9c12a9838d417)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.388318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6383d135-7fe4-47e2-937e-354e680ea773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110edd64c1d34eeca08390b8573e8d49'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.390111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.402263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:05.417194 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3398/6871] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:05 2022] DELETE /identity/v3/users/110edd64c1d34eeca08390b8573e8d49 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:05.421482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.421963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.422431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.446594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:05.448277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.456910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff6a52231e946ba9aa9c12a9838d417', 'request_id': 'req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.463947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff6a52231e946ba9aa9c12a9838d417', 'request_id': 'req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.471608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff6a52231e946ba9aa9c12a9838d417', 'request_id': 'req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.490398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] The token cache is being invalidate because project bff6a52231e946ba9aa9c12a9838d417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:05.491474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.491762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5aeb50ae-7b26-408d-8a0b-6e4b03405c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:05.493278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3474/6872] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:05 2022] DELETE /identity/v3/projects/bff6a52231e946ba9aa9c12a9838d417 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:05.499995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103be8-f546-44a8-87d8-49f982f4fc45 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.500284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103be8-f546-44a8-87d8-49f982f4fc45 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.500564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00103be8-f546-44a8-87d8-49f982f4fc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.522571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3399/6873] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:05.522792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-00103be8-f546-44a8-87d8-49f982f4fc45 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:05.523606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-00103be8-f546-44a8-87d8-49f982f4fc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.530975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.532315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.538678 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3475/6874] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:05.542832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.543243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.544054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.571049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:05.572339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.587328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3476/6875] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:05.589154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.599586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08652-5dab-4c3e-a789-dce4cb544841 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.602866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e07cdc-aa4e-4507-8c45-4d409a358958 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.604625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.605222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.605500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.605772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.609668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.610284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.610599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.610874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.611523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:05.611907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.613684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.615707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.617218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.630296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3477/6876] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:05.630483 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3400/6877] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:05.642356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.656231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176e004-b926-49a4-8b40-537e30b484e2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.661348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.661958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.662239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.662529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.663301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.665241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.700095 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3478/6878] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:05 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:05.705284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.717194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe7a7b-6d55-4340-b24b-d8e7c1fa9dab service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.798720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.799332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.799613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.799889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.800521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.802649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.826923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3401/6879] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:05 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:05.831418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.843517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517f29e-370f-4024-8140-0efe0b34e9d6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.848471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.849097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.849377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.849652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.850287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.851922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.862429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3479/6880] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:05.866522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.876959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bbcd5-ec13-4102-a33e-9c79d714542c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.881862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.882465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.882783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.883056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.883720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.886006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.888422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:05.898547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3402/6881] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:05.903458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.914044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa38f4a-07e0-4fb0-845a-0784263d6b66 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:05.919186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:05.919837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:05.920490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:05.920490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] PATH_INFO: `/v3/projects/93b7cc018df043a9a7a92006aea3a23d/users/720ee51186ca41bdb33ebfad0889b65b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:05.936662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b7cc018df043a9a7a92006aea3a23d, user_id=720ee51186ca41bdb33ebfad0889b65b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:05.939492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74db0c-a6c9-4474-82b0-205fe9a56bd1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:05.951867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:05.954005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:05.954599 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3480/6882] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:05 2022] PUT /identity/v3/projects/93b7cc018df043a9a7a92006aea3a23d/users/720ee51186ca41bdb33ebfad0889b65b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:05.976371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed1f82-551e-43d9-8930-a8836de8524f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.051916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.052679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.053032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.053380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] PATH_INFO: `/v3/projects/1d9cbff9c0c5434eab9f681a4f94de5d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.054918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9cbff9c0c5434eab9f681a4f94de5d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.056886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.065293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.065832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.066563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.099748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] MFA Rules not processed for user `720ee51186ca41bdb33ebfad0889b65b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:06.101165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.114998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9cbff9c0c5434eab9f681a4f94de5d', 'request_id': 'req-53087d64-e230-4825-8277-d659a6288d81'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.127498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9cbff9c0c5434eab9f681a4f94de5d', 'request_id': 'req-53087d64-e230-4825-8277-d659a6288d81'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.136953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9cbff9c0c5434eab9f681a4f94de5d', 'request_id': 'req-53087d64-e230-4825-8277-d659a6288d81'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.157481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] The token cache is being invalidate because project 1d9cbff9c0c5434eab9f681a4f94de5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:06.157781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.158609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.160387 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3403/6883] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:05 2022] DELETE /identity/v3/projects/1d9cbff9c0c5434eab9f681a4f94de5d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:06.164038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.166040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.192150 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3481/6884] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:06.200077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.201454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.206514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:06.243788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53087d64-e230-4825-8277-d659a6288d81 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.249393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.250020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.250308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.250688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] PATH_INFO: `/v3/users/09befe431f3446b8ae6854e92f629a97` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.251853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09befe431f3446b8ae6854e92f629a97)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.253842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.272931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50c07-f87d-42dd-b2eb-4d9b5e1e64d6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.276890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.277762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.278042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.278316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.287924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.289765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.293280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:06.300339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3482/6885] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:06.310180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09befe431f3446b8ae6854e92f629a97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.310883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09befe431f3446b8ae6854e92f629a97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.318303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09befe431f3446b8ae6854e92f629a97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.326621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09befe431f3446b8ae6854e92f629a97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.327315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-912d25e3-2462-4456-95bf-da4a515d723a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09befe431f3446b8ae6854e92f629a97'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.328873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.330341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.344663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab28fa9-dfb0-4c70-8591-d042f8638f54 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.345147 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3404/6886] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:06 2022] DELETE /identity/v3/users/09befe431f3446b8ae6854e92f629a97 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:06.419682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.420412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.420685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.420955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] PATH_INFO: `/v3/users/720ee51186ca41bdb33ebfad0889b65b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.422985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] RBAC: Authorizing `identity:delete_user(user_id=720ee51186ca41bdb33ebfad0889b65b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.424977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.478038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720ee51186ca41bdb33ebfad0889b65b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.479335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720ee51186ca41bdb33ebfad0889b65b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.486680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720ee51186ca41bdb33ebfad0889b65b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.500194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720ee51186ca41bdb33ebfad0889b65b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.500895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720ee51186ca41bdb33ebfad0889b65b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.517797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3483/6887] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:06 2022] DELETE /identity/v3/users/720ee51186ca41bdb33ebfad0889b65b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:06.519213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.519506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.519784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.541223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:06.542675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.606906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3405/6888] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:06.616551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.618019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.653731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:06.658587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.660240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.677620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a8f3f-f667-473f-ae7c-93540d0b7d7c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.682741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.683382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.683663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.683939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] PATH_INFO: `/v3/projects/552c3413815d4b558bf99434a5fbecca` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.685087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] RBAC: Authorizing `identity:delete_project(project_id=552c3413815d4b558bf99434a5fbecca)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.687082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.693677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a121c2b-0187-4bbf-b6bf-c4a2f4b1723a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.698750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.699359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.699638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.699912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.709385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.710853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e877c-188a-4bd9-9474-a4bb692bb19f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.722261 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3484/6889] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:06.738809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552c3413815d4b558bf99434a5fbecca', 'request_id': 'req-2d18dc6a-8a94-4fa5-a726-706086b4261a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.745337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552c3413815d4b558bf99434a5fbecca', 'request_id': 'req-2d18dc6a-8a94-4fa5-a726-706086b4261a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.753466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552c3413815d4b558bf99434a5fbecca', 'request_id': 'req-2d18dc6a-8a94-4fa5-a726-706086b4261a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.765728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] The token cache is being invalidate because project 552c3413815d4b558bf99434a5fbecca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:06.766011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.766719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d18dc6a-8a94-4fa5-a726-706086b4261a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:06.768253 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3406/6890] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:06 2022] DELETE /identity/v3/projects/552c3413815d4b558bf99434a5fbecca => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:06.773479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.773931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.774268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.796389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:06.797598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.860578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3485/6891] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:06.869429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.869889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.870329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.893902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:06.895183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:06.904580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3407/6892] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:06.908908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.919336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf13de-72a1-4a56-942f-24747753d990 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.924015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.924623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.924901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.925172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.925839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:06.926119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.927893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.941134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3486/6893] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:06.945434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:06.955378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719b97b-281e-484a-8a3f-b932c405d1b3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:06.960320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:06.960927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:06.961207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:06.961482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:06.962109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:06.965081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:06.992630 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3408/6894] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:06.998002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.010963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.013473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.014660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc79b6c-f548-488b-8def-e8bd905049c0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.019886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:07.084227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.084900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.085188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.085489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.086158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.088229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.088687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df73b-ec0c-46b7-84b7-80ac3f155d90 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.093956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.094635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.094916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.095190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] PATH_INFO: `/v3/projects/93b7cc018df043a9a7a92006aea3a23d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.096321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b7cc018df043a9a7a92006aea3a23d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.098223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.114001 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3487/6895] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:06 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:07.115002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.125654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c5372-64d3-4cee-8a06-8a7233a390cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.131101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.131699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.131983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.132251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.132943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.134600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.146868 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3488/6896] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:07.150960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.151181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b7cc018df043a9a7a92006aea3a23d', 'request_id': 'req-2de3ad5f-c555-4af5-9a17-06bb645850f0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:07.157152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b7cc018df043a9a7a92006aea3a23d', 'request_id': 'req-2de3ad5f-c555-4af5-9a17-06bb645850f0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:07.164490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ea3cc-f5c7-4b41-94f8-5610e13bda88 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.164878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b7cc018df043a9a7a92006aea3a23d', 'request_id': 'req-2de3ad5f-c555-4af5-9a17-06bb645850f0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:07.176602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] The token cache is being invalidate because project 93b7cc018df043a9a7a92006aea3a23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:07.176892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:07.177550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2de3ad5f-c555-4af5-9a17-06bb645850f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:07.179084 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3409/6897] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:07 2022] DELETE /identity/v3/projects/93b7cc018df043a9a7a92006aea3a23d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:07.184271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f4d0-40c8-4041-b106-89087126726e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.184564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f4d0-40c8-4041-b106-89087126726e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.184834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f4d0-40c8-4041-b106-89087126726e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.196538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:07.206047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-34a2f4d0-40c8-4041-b106-89087126726e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:07.207389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-34a2f4d0-40c8-4041-b106-89087126726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.229625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.230602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.230890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.231169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.231813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.233424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.250046 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3489/6898] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:07.253743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.255514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.268713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338d896-c90c-4a19-8297-2a78851d1c9d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.279336 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3410/6899] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:07.280581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.281271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.281546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.281640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.281994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.282099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] PATH_INFO: `/v3/projects/daad36493f4e4c2890a0fffac9910ed8/users/c9cdc7e12a914f8c9a16436e70d23695/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.282401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.304099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daad36493f4e4c2890a0fffac9910ed8, user_id=c9cdc7e12a914f8c9a16436e70d23695, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.305374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:07.307322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.309031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6afeb4-50ca-4e0b-8b62-7007efc861f6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.319928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3411/6900] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:07.321103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.331711 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3490/6901] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:07 2022] PUT /identity/v3/projects/daad36493f4e4c2890a0fffac9910ed8/users/c9cdc7e12a914f8c9a16436e70d23695/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:07.333416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6ecbd-2289-4e2f-a377-431d81932137 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.342598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.342889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.343163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.362533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:07.378148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] MFA Rules not processed for user `c9cdc7e12a914f8c9a16436e70d23695`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:07.379402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.406366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.407408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.407782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.408130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.408855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:07.409230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.411347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.433542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3412/6902] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:07.437900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.452326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad75985-f9a7-45b2-bed9-1140f63615b5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.457808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.458409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.458721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.458996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.459854 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3491/6903] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:07.459982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.461794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.464425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.477364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e99d3b-2804-4b49-a658-444601da09d6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.483695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.484330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.484927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.485447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.486342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.488562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.498070 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3413/6904] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:07.499168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.517385 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3492/6905] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:07.521952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.563735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90880410-39d0-4694-bdac-68b9c4129669 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.591030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33703-8e7e-4c58-9b94-23b81a47136f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.596230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.596832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.597108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.597374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.597991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.599996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.622619 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3493/6906] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:07.626677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.627159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.627241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.627491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.627759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.628429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.634575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.634779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:07.639171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65fd92e-f289-4514-9484-ae5fef47e642 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.644766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.645428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.646200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.646316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.647081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.649054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.659291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3414/6907] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:07.659698 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3494/6908] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:07.660353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.663546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.672075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed211b-bf58-467d-9023-bc356f9fb2c1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.673767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164343fd-542c-4cf7-ad9c-936d74a1f1ab admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.676861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.677450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.677733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.678006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.678464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.678680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.679081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.679353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.679619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.680233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.680322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.682422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.696789 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3495/6909] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:07.697833 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3415/6910] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:07.698655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.701558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.708859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1c07-8c6e-4a51-9743-c86ee80331b6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.711498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7f0-8f58-41dd-a75a-290634a46305 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.714858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.715506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.715783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.716057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] PATH_INFO: `/v3/projects/627a1d5ae08a4fc08f142c415295af93/users/657f0cc1c6064614a6fb76970602b95e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.716976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.717554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.717829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.718101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.718775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.720627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.731211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=627a1d5ae08a4fc08f142c415295af93, user_id=657f0cc1c6064614a6fb76970602b95e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.732246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3416/6911] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:07.734159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143aab6f-eb09-4a63-aad0-bb394d243b2b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.736036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.747620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05a0d-c683-453b-9533-319fee9aa52c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.749474 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3496/6912] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:07 2022] PUT /identity/v3/projects/627a1d5ae08a4fc08f142c415295af93/users/657f0cc1c6064614a6fb76970602b95e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:07.757214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:07.760985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.761262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.761539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.790451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] MFA Rules not processed for user `657f0cc1c6064614a6fb76970602b95e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:07.792069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.819799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.820585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.820864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.821134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] PATH_INFO: `/v3/projects/19447dfe0d12474e9598c929c0d7ec86/users/6396251efc624dc58b599889eaac047d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.837188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19447dfe0d12474e9598c929c0d7ec86, user_id=6396251efc624dc58b599889eaac047d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.840204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fac8c-d3ee-445a-b398-df4065ad5c96 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.853586 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3417/6913] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:07 2022] PUT /identity/v3/projects/19447dfe0d12474e9598c929c0d7ec86/users/6396251efc624dc58b599889eaac047d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:07.858426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.858766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.859047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.866003 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3497/6914] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:07.871060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.878901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:07.880083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.881789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ff73b-cb68-4305-bd8c-3035ed046df5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:07.947079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:07.948758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:07.949099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:07.949429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:07.950120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:07.951706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3418/6915] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:07.952256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:07.962406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:07.964425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:07.985116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3498/6916] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:07.986529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.029782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:08.036392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5956d0b-fa91-434a-8338-a495cdb4d267 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.068421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c046b-60b8-49fe-a963-bd999fb7f32b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.075609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.076225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.076531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.076815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.077412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.079894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.099104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.099370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.099464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.099719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.105193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3499/6917] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:07 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:08.109141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.110547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:08.113115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.115593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.121827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b90d65-6337-460d-b642-1f0b2ba05cc8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.127918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.128540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.128815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.129028 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3419/6918] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:08.129396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.130048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.131775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.147623 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3500/6919] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:08.148867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.160756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a414-4209-4a01-af8a-7f2f28687d94 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.166954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.167619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.167981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.168328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] PATH_INFO: `/v3/projects/b6a20c0139c74a50ab65ef5ced494999/users/da2bbd72521f430e8770f4ef4efbbeb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.186638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a20c0139c74a50ab65ef5ced494999, user_id=da2bbd72521f430e8770f4ef4efbbeb1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.189501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.208772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3420/6920] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:08 2022] PUT /identity/v3/projects/b6a20c0139c74a50ab65ef5ced494999/users/da2bbd72521f430e8770f4ef4efbbeb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:08.213842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.228667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665a3a0-77cb-4210-be25-db2294604509 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.303423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.304147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.304325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.305010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.305289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.307200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a1278-2850-4b77-89dd-9a283cc68e8c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.326570 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3501/6921] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:08.329818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.345988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63426666-cccf-497c-91d5-6add175b4550 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.351559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.352298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.353192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.353192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] PATH_INFO: `/v3/projects/b6a20c0139c74a50ab65ef5ced494999/users/da2bbd72521f430e8770f4ef4efbbeb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.356506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a20c0139c74a50ab65ef5ced494999, user_id=da2bbd72521f430e8770f4ef4efbbeb1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.359091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.375254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3421/6922] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:08 2022] PUT /identity/v3/projects/b6a20c0139c74a50ab65ef5ced494999/users/da2bbd72521f430e8770f4ef4efbbeb1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:08.385455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.385816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.386094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.434801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] MFA Rules not processed for user `da2bbd72521f430e8770f4ef4efbbeb1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:08.437301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:08.578095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:08.580544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:08.583037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3502/6923] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:08.692538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b47a-97e4-4417-b5d4-3b28b53ac9f5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:08.697232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:08.698520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:08.698878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:08.699631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:08.712879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:08.714809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a865f0-700c-457f-a99d-a9cdffb758ad service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:08.734894 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3422/6924] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:09.367079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:09.368562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:09.384733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfff07-ff9c-4f19-9f9f-d2a8b2e06061 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:09.389576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:09.390192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:09.390471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:09.390786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:09.423404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:09.460735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:09.462218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:09.473092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3503/6925] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:13.024060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.024548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.024635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.057570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] MFA Rules not processed for user `6396251efc624dc58b599889eaac047d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:13.059278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:13.164018 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3423/6926] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:13.165798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.242836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e44a1-4d2f-4bbb-b2db-e9ff62b41f0b service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.248629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.249388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.249766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.250144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.250918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.252930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.283965 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3504/6927] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:13 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:13.288852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.302597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b6736-98bb-4f60-8f6a-d0c613cb1d6c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.372522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:13.389060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.389861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.390527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.390946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.391617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.393738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.418800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3424/6928] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:13 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:13.422711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.436525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a415f-887d-4ffb-8a35-ed690aaa03f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.444453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.445277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.445568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.445872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.446555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.448227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.463028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3505/6929] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:13.464181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.473620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.476832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543140d2-8605-4b55-b0c0-af62761bb9e1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.483241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.483926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.484207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.484482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] PATH_INFO: `/v3/projects/fa9ec105b82746a0ae350b05618b42fe/users/d7deb645f8a148a19089eff0c739bf35/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.486460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd907161-39e9-47d4-9f57-6cbd5c2652a4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.491372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.492185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.492556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.492924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] PATH_INFO: `/v3/users/c9cdc7e12a914f8c9a16436e70d23695` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.495364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9cdc7e12a914f8c9a16436e70d23695)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.497499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.502787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9ec105b82746a0ae350b05618b42fe, user_id=d7deb645f8a148a19089eff0c739bf35, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.505965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.522472 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3425/6930] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:13 2022] PUT /identity/v3/projects/fa9ec105b82746a0ae350b05618b42fe/users/d7deb645f8a148a19089eff0c739bf35/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:13.526838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.567070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9cdc7e12a914f8c9a16436e70d23695'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:13.568205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9cdc7e12a914f8c9a16436e70d23695'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:13.578588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9cdc7e12a914f8c9a16436e70d23695'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:13.596231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9cdc7e12a914f8c9a16436e70d23695'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:13.597004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-19314d13-aa4f-42c5-b821-47690e1ba9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9cdc7e12a914f8c9a16436e70d23695'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:13.613642 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3506/6931] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:13 2022] DELETE /identity/v3/users/c9cdc7e12a914f8c9a16436e70d23695 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:13.617654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.618643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.619664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.629135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c546-fd78-4c41-864f-04ba5394c3c3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.643987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:13.646368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:13.660775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:13.714967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.715578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.715849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.716120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.716867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.718615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.722851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3507/6932] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:13.734828 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3426/6933] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:13.736000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.737904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:13.747242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:13.750133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:13.753802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094bed7-c39d-49f9-83b8-daa92f2640b4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.764419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.765676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.766101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.766475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] PATH_INFO: `/v3/projects/fa9ec105b82746a0ae350b05618b42fe/users/d7deb645f8a148a19089eff0c739bf35/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:13.770027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9ec105b82746a0ae350b05618b42fe, user_id=d7deb645f8a148a19089eff0c739bf35, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:13.774120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:13.788701 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3508/6934] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:13 2022] PUT /identity/v3/projects/fa9ec105b82746a0ae350b05618b42fe/users/d7deb645f8a148a19089eff0c739bf35/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:13.832034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1fc0c-cdcb-44dd-909a-1a81962773d4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:13.847089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:13.900256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:13.900994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:13.901438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:13.901653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:14.001216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:14.003496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:14.021089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3427/6935] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:14.402973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:14.404977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:14.430350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9ec5b-b490-407c-8560-f78e946488f0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:14.437277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:14.438209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:14.438648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:14.439031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] PATH_INFO: `/v3/projects/daad36493f4e4c2890a0fffac9910ed8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:14.440980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daad36493f4e4c2890a0fffac9910ed8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:14.443267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:14.505722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daad36493f4e4c2890a0fffac9910ed8', 'request_id': 'req-55e55c34-a389-4293-9a02-0f7bb6398bb5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.515015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daad36493f4e4c2890a0fffac9910ed8', 'request_id': 'req-55e55c34-a389-4293-9a02-0f7bb6398bb5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.525617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daad36493f4e4c2890a0fffac9910ed8', 'request_id': 'req-55e55c34-a389-4293-9a02-0f7bb6398bb5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.551033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] The token cache is being invalidate because project daad36493f4e4c2890a0fffac9910ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:14.551252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.551944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.554880 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3509/6936] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:14 2022] DELETE /identity/v3/projects/daad36493f4e4c2890a0fffac9910ed8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:14.559103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:14.569589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fedf4-7219-4bd1-a0a7-aece7f8cef0f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:14.652814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:14.653474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:14.653909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:14.654193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] PATH_INFO: `/v3/users/657f0cc1c6064614a6fb76970602b95e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:14.657432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=657f0cc1c6064614a6fb76970602b95e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:14.659459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:14.716357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '657f0cc1c6064614a6fb76970602b95e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.717094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '657f0cc1c6064614a6fb76970602b95e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.726235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '657f0cc1c6064614a6fb76970602b95e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.735827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '657f0cc1c6064614a6fb76970602b95e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.736735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '657f0cc1c6064614a6fb76970602b95e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:14.742104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:14.770146 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3428/6937] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:14 2022] DELETE /identity/v3/users/657f0cc1c6064614a6fb76970602b95e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:15.501285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:15.502828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:15.533322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e55c34-a389-4293-9a02-0f7bb6398bb5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:15.672357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:15.673006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:15.673418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:15.673703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] PATH_INFO: `/v3/projects/627a1d5ae08a4fc08f142c415295af93` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:15.674893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=627a1d5ae08a4fc08f142c415295af93)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:15.676951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:15.731569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:15.752992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627a1d5ae08a4fc08f142c415295af93', 'request_id': 'req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.762291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627a1d5ae08a4fc08f142c415295af93', 'request_id': 'req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.772431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627a1d5ae08a4fc08f142c415295af93', 'request_id': 'req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.790394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] The token cache is being invalidate because project 627a1d5ae08a4fc08f142c415295af93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:15.790394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.791917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.796461 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3510/6938] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:15 2022] DELETE /identity/v3/projects/627a1d5ae08a4fc08f142c415295af93 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:15.797700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:15.799347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:15.819609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971f9d9-e395-4463-9d5e-a9bf101cf4ff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:15.897678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:15.898332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:15.898712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:15.899064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] PATH_INFO: `/v3/users/da2bbd72521f430e8770f4ef4efbbeb1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:15.900836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da2bbd72521f430e8770f4ef4efbbeb1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:15.903855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:15.976139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da2bbd72521f430e8770f4ef4efbbeb1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.977962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da2bbd72521f430e8770f4ef4efbbeb1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:15.991092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da2bbd72521f430e8770f4ef4efbbeb1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:16.001750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da2bbd72521f430e8770f4ef4efbbeb1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:16.003019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5b576b56-e2e3-43cc-a12f-cdedc71f2d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da2bbd72521f430e8770f4ef4efbbeb1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:16.027478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3429/6939] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:15 2022] DELETE /identity/v3/users/da2bbd72521f430e8770f4ef4efbbeb1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:17.179154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:17.181494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:17.210258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203f5da-3b46-45d9-afa9-3a7a1985c2e8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:17.364549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:17.365707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:17.382477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:17.383796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] PATH_INFO: `/v3/projects/b6a20c0139c74a50ab65ef5ced494999` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:17.385629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a20c0139c74a50ab65ef5ced494999)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:17.388738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:17.497992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a20c0139c74a50ab65ef5ced494999', 'request_id': 'req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:17.506411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a20c0139c74a50ab65ef5ced494999', 'request_id': 'req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:17.517581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a20c0139c74a50ab65ef5ced494999', 'request_id': 'req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:17.534155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] The token cache is being invalidate because project b6a20c0139c74a50ab65ef5ced494999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:17.534535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:17.535205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-54603fb3-a0e2-4963-9bbb-72cd26aaedfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:17.537202 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3511/6940] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:17 2022] DELETE /identity/v3/projects/b6a20c0139c74a50ab65ef5ced494999 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:17.543546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:17.544478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:17.544910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:17.576514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:17.577976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:17.733946 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3430/6941] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:17.741626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:17.741992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:17.742243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:17.787463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:17.788779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:17.808080 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3512/6942] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:17.812540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:17.821050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:17.829555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca96a07-6df5-494b-a1a7-40f384577d04 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:17.843147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:17.843973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:17.844738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:17.844933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:17.845991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:17.846427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:17.849519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:17.880766 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3431/6943] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:17.885834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:17.909941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e448bf-eabe-46c9-b73e-c2d9d29b9d81 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:17.925339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:17.925857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:17.926130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:17.926401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:17.927124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:17.929080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:18.031948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3513/6944] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:17 2022] POST /identity/v3/projects => generated 401 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:18.036905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:18.053228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d1eb4-e699-4ca3-bf4b-b48c6058a840 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:18.226093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:18.231815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:18.233865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:18.234307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:18.235177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:18.241871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:18.310159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:18.310706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3432/6945] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:18 2022] POST /identity/v3/users => generated 301 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:18.331204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce832cd-484d-4fc4-a5d2-cfafe4ab450c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:18.347737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:18.348658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:18.349369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:18.351062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:18.351964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:18.357375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:18.405310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3514/6946] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:18.406838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:18.445503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a03634-2e0b-4ef6-85fd-ae807681aa40 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:18.480067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:18.481582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:18.481582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:18.489026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:18.489832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:18.491744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee667f5-46a4-4ac7-8ec1-40214ba060ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:18.545493 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3433/6947] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:18.548700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:18.591027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07d603-d225-4ccf-9c60-05c05a46cce3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:18.608403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:18.623249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:18.623742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:18.626006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] PATH_INFO: `/v3/projects/22d42a70326942c5afdc2d685885a6da/users/dff92beb66ce43e28321bca7595a1528/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:18.650934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:18.686204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d42a70326942c5afdc2d685885a6da, user_id=dff92beb66ce43e28321bca7595a1528, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:18.692054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:18.730639 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3515/6948] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:18 2022] PUT /identity/v3/projects/22d42a70326942c5afdc2d685885a6da/users/dff92beb66ce43e28321bca7595a1528/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:18.739575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:18.740135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:18.740355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:18.795074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] MFA Rules not processed for user `dff92beb66ce43e28321bca7595a1528`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:18.796756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:18.981497 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3434/6949] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:18.985984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.148762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e689e-44ac-4c92-b57f-118f90923055 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.156319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.156947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.157225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.157527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.158292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.160853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.236801 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3516/6950] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:18 2022] POST /identity/v3/projects => generated 399 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:19.240810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.263105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30d14e-4beb-4cee-9d00-b89efb7a3240 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.376942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.380086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.380374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.381248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.381364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.384358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.392449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:19.421663 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3435/6951] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:19 2022] POST /identity/v3/users => generated 306 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:19.423200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.437961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e448e3b-007c-43f9-886a-a827638846f7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.450569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.451506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.451764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.452221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.453094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.455078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.470756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3517/6952] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:19.474717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.486899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05712d9c-d4a5-4e1a-a064-5636860a0a4b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.492691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.494391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.495052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.495323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] PATH_INFO: `/v3/projects/a81cfea681d64193a5cb11f4032230b2/users/f01b108bc9d74f449c1f4bef49cd6c92/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.514521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81cfea681d64193a5cb11f4032230b2, user_id=f01b108bc9d74f449c1f4bef49cd6c92, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.517971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.545937 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3436/6953] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:19 2022] PUT /identity/v3/projects/a81cfea681d64193a5cb11f4032230b2/users/f01b108bc9d74f449c1f4bef49cd6c92/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:19.549735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.571621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2488fb-f33a-4105-8022-2150528f5171 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.741617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.742933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.743471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.744599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.745799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.747684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02788d-2a52-47c7-a38a-96ed7fdc3bc3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.771850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3518/6954] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:19.776005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:19.804613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566152-0d2d-452a-b0db-f671937fed76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:19.817121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:19.817965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.818726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.825236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] PATH_INFO: `/v3/projects/a81cfea681d64193a5cb11f4032230b2/users/f01b108bc9d74f449c1f4bef49cd6c92/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:19.834777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a81cfea681d64193a5cb11f4032230b2, user_id=f01b108bc9d74f449c1f4bef49cd6c92, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:19.838061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:19.868158 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3437/6955] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:19 2022] PUT /identity/v3/projects/a81cfea681d64193a5cb11f4032230b2/users/f01b108bc9d74f449c1f4bef49cd6c92/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:19.879774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:19.880054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:19.880356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:20.017916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] MFA Rules not processed for user `f01b108bc9d74f449c1f4bef49cd6c92`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:20.024800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:20.169314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:20.250471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3519/6956] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:20.258860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:20.262198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:20.452121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e016e-82a8-47d7-a6ed-a11b68615e19 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:20.465619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:20.467598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:20.467891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:20.469352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:20.490657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:20.492846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f137b-1bf9-423a-830a-5520db388398 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:20.534263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3438/6957] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:20.676031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:20.678955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:20.710166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caeb20c-1648-4532-8c23-33f2d6778e84 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:20.719678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:20.720363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:20.720747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:20.721202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:20.850210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:20.852457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:20.867952 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3520/6958] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:21.638170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:21.644758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:21.645455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:21.679627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] MFA Rules not processed for user `d7deb645f8a148a19089eff0c739bf35`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:21.680780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:21.788020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3439/6959] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:21.803115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:21.806936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:21.962889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60ae55-fc5c-40a6-af78-2bff6ea8ece2 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:21.972956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:21.973892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:21.974173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:21.974442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:21.983408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:21.984999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:21.996202 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3521/6960] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:22.332637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:22.334171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:22.413972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:22.458258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0f2ec-84e0-4c47-8e87-d52c6bed6289 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:22.465191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:22.465906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:22.466332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:22.466724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:22.469683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:22.488442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:22.491794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:22.515915 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3440/6961] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:23.027768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:23.183702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a901f-dce3-4695-a273-28129cd12aa6 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:23.189704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:23.190365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:23.190696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:23.191013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] PATH_INFO: `/v3/users/dff92beb66ce43e28321bca7595a1528` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:23.192939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff92beb66ce43e28321bca7595a1528)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:23.195372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:23.205365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:23.283404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:23.287417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:23.309195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb0ee7-4137-4feb-9b59-f5de963deaea service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:23.316202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:23.317099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:23.317524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:23.317939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:23.320530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:23.320718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff92beb66ce43e28321bca7595a1528'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:23.322182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff92beb66ce43e28321bca7595a1528'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:23.332119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff92beb66ce43e28321bca7595a1528'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:23.347875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:23.349816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c465897-ba10-4026-8d09-c8f879a0401c service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:23.365538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff92beb66ce43e28321bca7595a1528'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:23.366606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff92beb66ce43e28321bca7595a1528'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:23.389942 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3441/6962] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:23.403357 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3522/6963] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:23 2022] DELETE /identity/v3/users/dff92beb66ce43e28321bca7595a1528 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:23.407781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:23.408247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:23.408726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:23.445100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:23.446689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:23.599233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3442/6964] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:23.614465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:23.616355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:23.774044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34915b-a57e-4829-ab9a-7f3793f7fa7e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:23.790353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:23.791521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:23.791861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:23.792095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:23.824443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:23.826043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:23.843297 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3523/6965] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:24.607077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:24.608528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:24.621305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98ea30-3880-43e7-bc2b-3c49ec3679c6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:24.626235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:24.626893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:24.627231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:24.627510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] PATH_INFO: `/v3/projects/22d42a70326942c5afdc2d685885a6da` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:24.629062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d42a70326942c5afdc2d685885a6da)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:24.631197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:24.711156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d42a70326942c5afdc2d685885a6da', 'request_id': 'req-5ca5b98a-2fc8-449a-b074-2e07d72d197d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.717743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d42a70326942c5afdc2d685885a6da', 'request_id': 'req-5ca5b98a-2fc8-449a-b074-2e07d72d197d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.728858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d42a70326942c5afdc2d685885a6da', 'request_id': 'req-5ca5b98a-2fc8-449a-b074-2e07d72d197d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.745223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] The token cache is being invalidate because project 22d42a70326942c5afdc2d685885a6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:24.745545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.746216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.748367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3443/6966] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:24 2022] DELETE /identity/v3/projects/22d42a70326942c5afdc2d685885a6da => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:24.752809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:24.763594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d000ee-6c62-42ae-9df9-e95a8250e03f service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:24.878820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:24.879591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:24.880011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:24.880281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] PATH_INFO: `/v3/users/f01b108bc9d74f449c1f4bef49cd6c92` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:24.881479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01b108bc9d74f449c1f4bef49cd6c92)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:24.883473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:24.923027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:24.957474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01b108bc9d74f449c1f4bef49cd6c92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.958480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01b108bc9d74f449c1f4bef49cd6c92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:24.979441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01b108bc9d74f449c1f4bef49cd6c92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:25.061838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01b108bc9d74f449c1f4bef49cd6c92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:25.062967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3258a9ca-b634-4c96-9a39-1318e85f01cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01b108bc9d74f449c1f4bef49cd6c92'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:25.107433 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3524/6967] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:24 2022] DELETE /identity/v3/users/f01b108bc9d74f449c1f4bef49cd6c92 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:26.227209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:26.237153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:26.273062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5b98a-2fc8-449a-b074-2e07d72d197d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:26.339373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:26.380056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:26.381148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:26.382866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:26.383428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] PATH_INFO: `/v3/projects/a81cfea681d64193a5cb11f4032230b2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:26.386191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a81cfea681d64193a5cb11f4032230b2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:26.391957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:26.508500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a81cfea681d64193a5cb11f4032230b2', 'request_id': 'req-148c477c-9035-462b-84ee-49b78658fdf0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:26.520708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a81cfea681d64193a5cb11f4032230b2', 'request_id': 'req-148c477c-9035-462b-84ee-49b78658fdf0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:26.534690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a81cfea681d64193a5cb11f4032230b2', 'request_id': 'req-148c477c-9035-462b-84ee-49b78658fdf0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:26.565273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] The token cache is being invalidate because project a81cfea681d64193a5cb11f4032230b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:26.565567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:26.568741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-148c477c-9035-462b-84ee-49b78658fdf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:26.572394 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3444/6968] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:26 2022] DELETE /identity/v3/projects/a81cfea681d64193a5cb11f4032230b2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:26.577400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:26.578251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:26.578957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:26.655722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:26.657826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:26.770987 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3525/6969] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:26.776114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:26.776265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:26.776550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:26.821012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:26.824627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:26.846392 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3445/6970] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:26.851429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:26.864389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac551d-c29c-4ee4-b7a8-c08b63a59161 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:26.871275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:26.872347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:26.872850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:26.874080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:26.875120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:26.875669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:26.877817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:26.893137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3526/6971] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:26.897796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:26.912670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc31044-b574-4bf4-a06e-561a515ea655 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:26.922559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:26.923190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:26.923617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:26.924055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:26.924719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:26.934240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:26.989660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3446/6972] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:26 2022] POST /identity/v3/projects => generated 409 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:26.994069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.009859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9c720-a07f-4c2e-99dc-7422ca2addd0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.201633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.202773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.203261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.203781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.205139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.210718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.245214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3527/6973] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:26 2022] POST /identity/v3/users => generated 305 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:27.249351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.261429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c16a8-0c91-49fc-aa15-62611c259e6c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.282039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.282627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.283235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.283459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.284435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.286464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.305180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.314724 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3447/6974] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:27.320430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480c93-0f45-4dcb-a9e2-c1a3a5fe3780 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.329446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.330560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.331125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.331649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.332549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.334882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeb717-ca18-43ff-a941-49d2fee4925e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.355001 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3528/6975] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:27.356544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.372765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f0688-a8bd-4c49-acfd-20a70a664875 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.383048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.384795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.385571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.386077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] PATH_INFO: `/v3/projects/b58f1b4d79bb4361b62530a5cf292536/users/9ff71bd40de948bfa275c959f6bab2df/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.410159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58f1b4d79bb4361b62530a5cf292536, user_id=9ff71bd40de948bfa275c959f6bab2df, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.413965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.463108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3448/6976] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:27 2022] PUT /identity/v3/projects/b58f1b4d79bb4361b62530a5cf292536/users/9ff71bd40de948bfa275c959f6bab2df/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:27.466316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.466767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.467289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.519288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:27.534067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] MFA Rules not processed for user `9ff71bd40de948bfa275c959f6bab2df`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:27.535447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:27.666608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3529/6977] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:27.670665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.749704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6bb33-af41-459a-b8d8-cdea7c0f4f85 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.754999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.755726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.756513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.756886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.757794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.760533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.780399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:27.805359 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3449/6978] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:27 2022] POST /identity/v3/projects => generated 409 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:27.809083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.820161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3739273-7a48-45c7-ba53-163373d4fc0a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.901523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.904038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.904336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.904615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.905320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.907583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.941292 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3530/6979] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:27 2022] POST /identity/v3/users => generated 311 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:27.945239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:27.959595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5fb46-783f-411a-ac95-61dd1c849edb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:27.973106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:27.973740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:27.974024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:27.974617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:27.976306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:27.977492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:27.995775 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3450/6980] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:28.000384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:28.012462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35a587-1701-4e02-95ea-47078873ca50 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:28.018852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:28.019268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:28.019786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:28.020245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] PATH_INFO: `/v3/projects/9015293bac9f41c38831260559904632/users/5fead7529aea48839a2f9eba905bd00d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:28.044012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015293bac9f41c38831260559904632, user_id=5fead7529aea48839a2f9eba905bd00d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:28.046814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:28.068863 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3531/6981] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:27 2022] PUT /identity/v3/projects/9015293bac9f41c38831260559904632/users/5fead7529aea48839a2f9eba905bd00d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:28.073929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:28.085350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e178b-6686-47b4-a904-21b847151507 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:28.158462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:28.159184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:28.159537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:28.159878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:28.160587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:28.162246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c237719-cb88-4b95-8631-4ea73c2b3ead admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:28.184133 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3451/6982] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:28.186828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:28.200715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d56a7c-f48d-4d75-aa2a-4df774c73137 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:28.206882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:28.207753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:28.208139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:28.208492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] PATH_INFO: `/v3/projects/9015293bac9f41c38831260559904632/users/5fead7529aea48839a2f9eba905bd00d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:28.210716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015293bac9f41c38831260559904632, user_id=5fead7529aea48839a2f9eba905bd00d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:28.214328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:28.235637 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3532/6983] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:28 2022] PUT /identity/v3/projects/9015293bac9f41c38831260559904632/users/5fead7529aea48839a2f9eba905bd00d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:28.239937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:28.240292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:28.240503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:28.300790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] MFA Rules not processed for user `5fead7529aea48839a2f9eba905bd00d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:28.307297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:28.490046 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3452/6984] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:28.504985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:28.507096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:28.677263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cab6a-72ec-4a00-b326-281c6b97b420 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:28.693454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:28.694088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:28.694362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:28.694697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:28.706039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:28.718890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:28.720347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:28.739857 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3533/6985] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:28 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:29.123427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:29.145297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b5023-847d-48d8-941c-57b7dab4bf28 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:29.159676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:29.160312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:29.160685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:29.160856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:29.298448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:29.300867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:29.312958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:29.324801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3453/6986] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:31.655131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:31.739366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:31.740993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:31.745478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02847d55-d64f-40b0-88b9-d5a6e64a344e service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:31.751471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:31.754140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:31.754424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:31.754744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] PATH_INFO: `/v3/users/9ff71bd40de948bfa275c959f6bab2df` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:31.756139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff71bd40de948bfa275c959f6bab2df)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:31.758558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:31.779687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87298866-f7ff-46a5-95f6-366bdaf4775e service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:31.785457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:31.787043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:31.787374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:31.787719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] PATH_INFO: `/v3/users/82a854bf2b0042439e87b8aa0476b0e4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:31.789130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a854bf2b0042439e87b8aa0476b0e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:31.792089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:31.823618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff71bd40de948bfa275c959f6bab2df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.824469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff71bd40de948bfa275c959f6bab2df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.835714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff71bd40de948bfa275c959f6bab2df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.861521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff71bd40de948bfa275c959f6bab2df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.863533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2692bb40-2950-4320-8094-ac5fa21e8900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff71bd40de948bfa275c959f6bab2df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.875549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a854bf2b0042439e87b8aa0476b0e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.877289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a854bf2b0042439e87b8aa0476b0e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.878075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3534/6987] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:31 2022] DELETE /identity/v3/users/9ff71bd40de948bfa275c959f6bab2df => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:31.882763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:31.882924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:31.883338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:31.889509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a854bf2b0042439e87b8aa0476b0e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.903554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a854bf2b0042439e87b8aa0476b0e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.904703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a854bf2b0042439e87b8aa0476b0e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:31.909617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:31.911044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:31.934601 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3454/6988] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:31 2022] DELETE /identity/v3/users/82a854bf2b0042439e87b8aa0476b0e4 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:31.990280 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3535/6989] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:32.006782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:32.009518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:32.103554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ecebd-f565-4325-9d0e-f623d7c4d226 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:32.112366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:32.113609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:32.114616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:32.115064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:32.193594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:32.195111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:32.207264 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3455/6990] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:32.558300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:32.560223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:32.573923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1ce06-343f-4e64-9183-26c3bbba1edf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:32.579691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:32.580600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:32.580895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:32.581161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] PATH_INFO: `/v3/projects/bcdcaa646ba54f75abea17bdd8a6a62d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:32.582455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdcaa646ba54f75abea17bdd8a6a62d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:32.584720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:32.596005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:32.656018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdcaa646ba54f75abea17bdd8a6a62d', 'request_id': 'req-32bfa065-ac40-4255-89c2-de42314787b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.663312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdcaa646ba54f75abea17bdd8a6a62d', 'request_id': 'req-32bfa065-ac40-4255-89c2-de42314787b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.675482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdcaa646ba54f75abea17bdd8a6a62d', 'request_id': 'req-32bfa065-ac40-4255-89c2-de42314787b0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.694685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] The token cache is being invalidate because project bcdcaa646ba54f75abea17bdd8a6a62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:32.694899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.695693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.697675 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3536/6991] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:32 2022] DELETE /identity/v3/projects/bcdcaa646ba54f75abea17bdd8a6a62d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:32.701726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:32.713919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa0ebd-f5e6-4827-b43f-e19e7bd400af service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:32.736019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:32.787522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:32.788268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:32.788960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:32.788960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] PATH_INFO: `/v3/users/212241b715754f02a85a0e0de7cca15c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:32.790113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] RBAC: Authorizing `identity:delete_user(user_id=212241b715754f02a85a0e0de7cca15c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:32.792405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:32.848241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212241b715754f02a85a0e0de7cca15c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.848939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212241b715754f02a85a0e0de7cca15c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.859725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212241b715754f02a85a0e0de7cca15c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.870002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212241b715754f02a85a0e0de7cca15c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.874882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212241b715754f02a85a0e0de7cca15c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:32.895367 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3456/6992] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:32 2022] DELETE /identity/v3/users/212241b715754f02a85a0e0de7cca15c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:33.152495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.154206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.182884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfa065-ac40-4255-89c2-de42314787b0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.332657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.337267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.342571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.343650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.344090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.344829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] PATH_INFO: `/v3/projects/0117cd4ebb924451a1388a45aeac74e6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.346331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0117cd4ebb924451a1388a45aeac74e6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.353194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.364541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c55d3-d2cb-49d0-b2dd-69253cb3895b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.377792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.378761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.379613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.380201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] PATH_INFO: `/v3/projects/b58f1b4d79bb4361b62530a5cf292536` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.381531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58f1b4d79bb4361b62530a5cf292536)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.386546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.422539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0117cd4ebb924451a1388a45aeac74e6', 'request_id': 'req-15953fab-3da6-4b51-94f0-4d48b7944912'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.429828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0117cd4ebb924451a1388a45aeac74e6', 'request_id': 'req-15953fab-3da6-4b51-94f0-4d48b7944912'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.440698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0117cd4ebb924451a1388a45aeac74e6', 'request_id': 'req-15953fab-3da6-4b51-94f0-4d48b7944912'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.463983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] The token cache is being invalidate because project 0117cd4ebb924451a1388a45aeac74e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:33.464629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.465511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-15953fab-3da6-4b51-94f0-4d48b7944912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.467181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58f1b4d79bb4361b62530a5cf292536', 'request_id': 'req-1e6f16e4-2176-45d6-b579-543d4c47bc15'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.468370 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3537/6993] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:33 2022] DELETE /identity/v3/projects/0117cd4ebb924451a1388a45aeac74e6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:33.474176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc0cd-0ed9-4df5-830e-d336409657b8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.474749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc0cd-0ed9-4df5-830e-d336409657b8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.476283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fc0cd-0ed9-4df5-830e-d336409657b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.478869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58f1b4d79bb4361b62530a5cf292536', 'request_id': 'req-1e6f16e4-2176-45d6-b579-543d4c47bc15'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.488156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58f1b4d79bb4361b62530a5cf292536', 'request_id': 'req-1e6f16e4-2176-45d6-b579-543d4c47bc15'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.501615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-836fc0cd-0ed9-4df5-830e-d336409657b8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:33.503124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-836fc0cd-0ed9-4df5-830e-d336409657b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.503958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] The token cache is being invalidate because project b58f1b4d79bb4361b62530a5cf292536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:33.504259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.505087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.506806 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3457/6994] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:33 2022] DELETE /identity/v3/projects/b58f1b4d79bb4361b62530a5cf292536 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:33.510552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.512686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.537839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:33.551031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:33.594618 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3538/6995] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:33.601481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.601843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.602889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.604576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f16e4-2176-45d6-b579-543d4c47bc15 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.609444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.610100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.610381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.610758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] PATH_INFO: `/v3/users/5fead7529aea48839a2f9eba905bd00d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.612235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fead7529aea48839a2f9eba905bd00d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.615243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.626642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:33.628184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.646022 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3539/6996] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:33.651018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.663968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311286f2-294f-4f7d-871c-9f08e22f2284 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.670362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.671205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.671569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.671984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.672729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:33.673179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.675233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.688489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fead7529aea48839a2f9eba905bd00d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.689240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fead7529aea48839a2f9eba905bd00d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.690693 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3540/6997] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:33.695121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.697445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:33.701070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fead7529aea48839a2f9eba905bd00d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.710595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fead7529aea48839a2f9eba905bd00d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.710991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6849d-a334-47e2-8565-269e816df339 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.711281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fead7529aea48839a2f9eba905bd00d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:33.717652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.718529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.718992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.719469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.720301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.722761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.736506 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3458/6998] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:33 2022] DELETE /identity/v3/users/5fead7529aea48839a2f9eba905bd00d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:33.764517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3541/6999] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:33 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:33.768626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.854639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054d955-0336-4303-9087-82070d6607a6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:33.931322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:33.948402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:33.949215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:33.949569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:33.949907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:33.950822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:33.952884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:33.981243 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3459/7000] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:33 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:33.985322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:33.997823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6853-73ca-4b3f-9cb3-19741c74aeb5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.003309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.005246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.005246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.005246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.005921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.007463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.019012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3542/7001] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:34.023193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.034866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62442-c612-4fef-a4a3-91923005812a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.041428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.042544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.042769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.043144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.043894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.045938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.060088 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3460/7002] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:34.064163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.078491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.081397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d96b6a-8cc1-4e9a-98aa-e216b5bc48b3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.082481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.091031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.091782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.091943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.092224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] PATH_INFO: `/v3/projects/da09a129f40347fe835f1acf608577bf/users/d3c69e0344ed42cbb45428592b144550/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.115830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da09a129f40347fe835f1acf608577bf, user_id=d3c69e0344ed42cbb45428592b144550, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.119942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e42946-8761-4d78-b8be-be075fe16f7c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.123928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0255e42-c39d-4387-b5b4-97f1f0fd068d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.131403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.132148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.132521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.134330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] PATH_INFO: `/v3/projects/9015293bac9f41c38831260559904632` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.135902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9015293bac9f41c38831260559904632)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.137909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.152816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3543/7003] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:34 2022] PUT /identity/v3/projects/da09a129f40347fe835f1acf608577bf/users/d3c69e0344ed42cbb45428592b144550/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:34.154656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.154800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.155088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.252728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] MFA Rules not processed for user `d3c69e0344ed42cbb45428592b144550`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:34.253995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.261021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9015293bac9f41c38831260559904632', 'request_id': 'req-57d5399a-1d34-4930-be29-524a58c07bf4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.280030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9015293bac9f41c38831260559904632', 'request_id': 'req-57d5399a-1d34-4930-be29-524a58c07bf4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.310489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9015293bac9f41c38831260559904632', 'request_id': 'req-57d5399a-1d34-4930-be29-524a58c07bf4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.343362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] The token cache is being invalidate because project 9015293bac9f41c38831260559904632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:34.343662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.344485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-57d5399a-1d34-4930-be29-524a58c07bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.351762 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3461/7004] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:34 2022] DELETE /identity/v3/projects/9015293bac9f41c38831260559904632 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:34.371013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12b8-7fda-4c5c-966f-73673925875b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.371674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12b8-7fda-4c5c-966f-73673925875b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.372107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be12b8-7fda-4c5c-966f-73673925875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.416067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e1be12b8-7fda-4c5c-966f-73673925875b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:34.417360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e1be12b8-7fda-4c5c-966f-73673925875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.429863 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3544/7005] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:34.444054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.446374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.504452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:34.532317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342128-9602-47b6-8240-52c3ecb2a279 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.536149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3462/7006] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:34.537888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.539166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.540205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.540965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.545862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.546274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.546697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.555280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.557513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.572649 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3545/7007] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:34.588243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:34.590129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.600305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.602607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.608183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3463/7008] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:34.612375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.620516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc882b09-6f2f-4dba-8bb0-97d26de7329c service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.625606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020155-98b8-4a74-9f5a-9ba006fc3e42 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.627072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.627749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.628163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.628511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] PATH_INFO: `/v3/users/d3c69e0344ed42cbb45428592b144550` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.629836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c69e0344ed42cbb45428592b144550)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.632039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.657310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.658099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.658388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.658720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.659380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:34.660173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.661806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.684536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3464/7009] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:34.691922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.715622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf94bd-398d-46fe-a4af-46ab6e204a72 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:34.729678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:34.730317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.730721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.731084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.731557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:34.742086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:34.752251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c69e0344ed42cbb45428592b144550'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.754122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c69e0344ed42cbb45428592b144550'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.771829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c69e0344ed42cbb45428592b144550'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.809096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c69e0344ed42cbb45428592b144550'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.810305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-980fe1ad-eba3-4072-aa70-76ddc383d4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c69e0344ed42cbb45428592b144550'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:34.817042 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3465/7010] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:34.818758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:34.823168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.849531 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3546/7011] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:34 2022] DELETE /identity/v3/users/d3c69e0344ed42cbb45428592b144550 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:34.853222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:34.855597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:34.855953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:34.856356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:34.903206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:34.906634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:34.925564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2351-7e28-4866-b9a8-00d813adf548 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.027537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.028478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.028940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.029414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.030662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.042739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.060842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3547/7012] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:35.082176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.084006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:35.089013 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3466/7013] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:34 2022] POST /identity/v3/users => generated 286 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:35.105364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.125796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4685c-adb7-40fd-bdf4-c8f3d164e7a8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.139577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.139577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.139577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.139577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.140176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.140176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.157946 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3467/7014] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:35.159409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.172299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f02f3e-e3da-455e-a2f6-be609da01874 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.178330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.179066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.180287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.180287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.181249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.183012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.219436 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3468/7015] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:35.219436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.229810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d5589-5422-4722-9b4b-dede1c7d64ff None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.238690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05816b2-d2ec-48e3-a582-cbfcae76f427 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.239586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.240285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.240721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.241534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.251330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.252015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.252296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.252575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] PATH_INFO: `/v3/projects/834efd75e63140fabdf1549f7a10f035/users/4c2523aff0f14cd2a3b911470dcbfbc4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.262873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.264352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323dd81-4bad-431f-ac60-361133b3ca40 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.277580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:35.290560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3548/7016] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:35.292060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834efd75e63140fabdf1549f7a10f035, user_id=4c2523aff0f14cd2a3b911470dcbfbc4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.295555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.339233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3469/7017] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:35 2022] PUT /identity/v3/projects/834efd75e63140fabdf1549f7a10f035/users/4c2523aff0f14cd2a3b911470dcbfbc4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:35.343926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.345383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.345877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.411383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] MFA Rules not processed for user `4c2523aff0f14cd2a3b911470dcbfbc4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:35.412973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:35.505282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.509291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:35.526659 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3549/7018] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:35.540586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.542547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:35.657186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a57d5-208a-4ab5-9119-60f3ff0fd358 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.675967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.676600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.676899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.677135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.692565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.696554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.708600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be390a-6329-492c-a4c7-d438096dd661 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.714604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3550/7019] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:35.718228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.718885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.719161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.719431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] PATH_INFO: `/v3/users/6396251efc624dc58b599889eaac047d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.720590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] RBAC: Authorizing `identity:delete_user(user_id=6396251efc624dc58b599889eaac047d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.722830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.739329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.742467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:35.759935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a471e-552b-4992-ab71-fd88f0be7881 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:35.771590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:35.776550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.777603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.777824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] PATH_INFO: `/v3/users/4c2523aff0f14cd2a3b911470dcbfbc4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:35.779458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2523aff0f14cd2a3b911470dcbfbc4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:35.783407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:35.818900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6396251efc624dc58b599889eaac047d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.819712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6396251efc624dc58b599889eaac047d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.841282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6396251efc624dc58b599889eaac047d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.851178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6396251efc624dc58b599889eaac047d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.853608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6396251efc624dc58b599889eaac047d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.908162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2523aff0f14cd2a3b911470dcbfbc4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.910523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2523aff0f14cd2a3b911470dcbfbc4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.912880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3470/7020] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:35 2022] DELETE /identity/v3/users/6396251efc624dc58b599889eaac047d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:35.913947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:35.918124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:35.925120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2523aff0f14cd2a3b911470dcbfbc4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.955153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2523aff0f14cd2a3b911470dcbfbc4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.956474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2a0bae9c-4805-4048-a427-e6c90fa9f5a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2523aff0f14cd2a3b911470dcbfbc4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:35.984198 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3551/7021] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:35 2022] DELETE /identity/v3/users/4c2523aff0f14cd2a3b911470dcbfbc4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:35.989017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:35.989472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:35.989920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.005536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:36.029450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:36.031367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.067480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56d1e9-c819-4569-bc49-e9f74343ebda admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.120287 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3552/7022] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:36.131311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:36.134384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.156688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.158742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.159138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.159530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] PATH_INFO: `/v3/projects/da09a129f40347fe835f1acf608577bf` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.161958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da09a129f40347fe835f1acf608577bf)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.164014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:36.231076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16700807-a02c-4644-bcd6-cfe1e891e2c0 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.235224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:36.236453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.237596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.238980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.239650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.271264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da09a129f40347fe835f1acf608577bf', 'request_id': 'req-240b8c04-ce60-46ea-9abf-efc02ace40e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.272037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.275217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:36.279300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da09a129f40347fe835f1acf608577bf', 'request_id': 'req-240b8c04-ce60-46ea-9abf-efc02ace40e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.291701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da09a129f40347fe835f1acf608577bf', 'request_id': 'req-240b8c04-ce60-46ea-9abf-efc02ace40e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.296594 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3553/7023] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:36.298168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:36.299841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.324033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ad84-8d99-421f-8b5c-41c271711ca1 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.329011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] The token cache is being invalidate because project da09a129f40347fe835f1acf608577bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:36.329304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.330265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-240b8c04-ce60-46ea-9abf-efc02ace40e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.334124 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3471/7024] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:35 2022] DELETE /identity/v3/projects/da09a129f40347fe835f1acf608577bf => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:36.338948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0fe0e-5954-4548-8fe6-6847b0942352 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.339275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0fe0e-5954-4548-8fe6-6847b0942352 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.343005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0fe0e-5954-4548-8fe6-6847b0942352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.387594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6da0fe0e-5954-4548-8fe6-6847b0942352 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:36.391417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6da0fe0e-5954-4548-8fe6-6847b0942352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.467498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.468378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.468661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.470702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] PATH_INFO: `/v3/projects/19447dfe0d12474e9598c929c0d7ec86` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.474025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19447dfe0d12474e9598c929c0d7ec86)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.476184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:36.565927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3472/7025] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:36.570406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.571094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.571620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.610292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:36.615745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.639828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19447dfe0d12474e9598c929c0d7ec86', 'request_id': 'req-99515515-ef44-49f0-9c75-68d78ad3c554'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.641491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:36.654650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19447dfe0d12474e9598c929c0d7ec86', 'request_id': 'req-99515515-ef44-49f0-9c75-68d78ad3c554'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.670800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19447dfe0d12474e9598c929c0d7ec86', 'request_id': 'req-99515515-ef44-49f0-9c75-68d78ad3c554'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.707026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] The token cache is being invalidate because project 19447dfe0d12474e9598c929c0d7ec86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:36.707521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.708423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:36.710968 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3554/7026] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:36 2022] DELETE /identity/v3/projects/19447dfe0d12474e9598c929c0d7ec86 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:36.715762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:36.717272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:36.784650 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3473/7027] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:36.789367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:36.811983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690debc-cd23-43fd-b5bb-aab2b83af295 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.824421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.826667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.828023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.829024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.832633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:36.832885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.835184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:36.848592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99515515-ef44-49f0-9c75-68d78ad3c554 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.870718 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3474/7028] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:36.870930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.870930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.870930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.870930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] PATH_INFO: `/v3/users/d7deb645f8a148a19089eff0c739bf35` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.871273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:36.871353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7deb645f8a148a19089eff0c739bf35)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.876211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:36.926029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b095673-bb78-422e-96c0-fa3a89460330 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:36.950145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:36.950400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:36.950642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:36.950890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:36.951528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:36.961495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.007500 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3475/7029] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:36 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:37.009638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.014632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.028350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7deb645f8a148a19089eff0c739bf35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.029040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7deb645f8a148a19089eff0c739bf35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.042354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7deb645f8a148a19089eff0c739bf35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.061070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:37.078828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7deb645f8a148a19089eff0c739bf35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.079896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7deb645f8a148a19089eff0c739bf35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.101658 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3555/7030] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:36 2022] DELETE /identity/v3/users/d7deb645f8a148a19089eff0c739bf35 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:37.103019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.104730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.131429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb912713-b061-431b-ae1c-3a0630b2fc76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.192108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:37.201606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c2b7-89b7-4f82-8352-8e20f1bee7e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.210259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.211360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.211970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.212322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.213121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.215591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.223515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.224161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.224449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.224726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] PATH_INFO: `/v3/projects/834efd75e63140fabdf1549f7a10f035` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.226213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834efd75e63140fabdf1549f7a10f035)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.229230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.283843 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3556/7031] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:37.287864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.306999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322afb8-e372-4dcd-93cb-b79d63ab58f9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.327530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.334827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.335220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.335568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.342548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.346375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.352856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834efd75e63140fabdf1549f7a10f035', 'request_id': 'req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.365506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834efd75e63140fabdf1549f7a10f035', 'request_id': 'req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.379264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3557/7032] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:37.379264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.380530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.394764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834efd75e63140fabdf1549f7a10f035', 'request_id': 'req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.413941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] The token cache is being invalidate because project 834efd75e63140fabdf1549f7a10f035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:37.414282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.415565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.417078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164156a3-12b3-465d-8c3a-36b9076865f7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.418591 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3476/7033] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:37 2022] DELETE /identity/v3/projects/834efd75e63140fabdf1549f7a10f035 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:37.419856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.421619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.517466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.518305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.518642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.518916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] PATH_INFO: `/v3/projects/fa9ec105b82746a0ae350b05618b42fe` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.522349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa9ec105b82746a0ae350b05618b42fe)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.526654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.527709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f01ec5-d255-4f9c-a7f9-18f0e4c82ac0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.533096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.533707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.534629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.534912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.535567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.537597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10224666-bce1-4bae-9f4d-6e104de11b8c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.540652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:37.550002 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3477/7034] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:37.554932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9413a4-9dfe-4611-9891-d7e6a5d912b6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.556373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9413a4-9dfe-4611-9891-d7e6a5d912b6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.557479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9413a4-9dfe-4611-9891-d7e6a5d912b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.581297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a9413a4-9dfe-4611-9891-d7e6a5d912b6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:37.582940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a9413a4-9dfe-4611-9891-d7e6a5d912b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.603423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa9ec105b82746a0ae350b05618b42fe', 'request_id': 'req-281f0785-b74c-484b-afe3-8c20e642fdc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.611730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa9ec105b82746a0ae350b05618b42fe', 'request_id': 'req-281f0785-b74c-484b-afe3-8c20e642fdc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.622972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa9ec105b82746a0ae350b05618b42fe', 'request_id': 'req-281f0785-b74c-484b-afe3-8c20e642fdc0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.647034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] The token cache is being invalidate because project fa9ec105b82746a0ae350b05618b42fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:37.648452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.650834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:37.654914 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3558/7035] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:37 2022] DELETE /identity/v3/projects/fa9ec105b82746a0ae350b05618b42fe => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:37.655823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.657563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.662751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3478/7036] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:37.664153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66ef5-2085-4c37-ba87-c27652d64e23 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.665199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66ef5-2085-4c37-ba87-c27652d64e23 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.665609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66ef5-2085-4c37-ba87-c27652d64e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.683231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f0785-b74c-484b-afe3-8c20e642fdc0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.688081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.688827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.689365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.689782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] PATH_INFO: `/v3/projects/04759b8d896645a08cf0eb979a5238c9/users/223e7daaa61e4799b9586e6f7f86f253/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.690878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f2d66ef5-2085-4c37-ba87-c27652d64e23 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:37.692318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f2d66ef5-2085-4c37-ba87-c27652d64e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.714964 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3479/7037] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:37.714964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae105-48a5-4d65-aaeb-b162dd4aed44 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.714964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae105-48a5-4d65-aaeb-b162dd4aed44 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.714964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ae105-48a5-4d65-aaeb-b162dd4aed44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.737571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04759b8d896645a08cf0eb979a5238c9, user_id=223e7daaa61e4799b9586e6f7f86f253, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.738067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-281ae105-48a5-4d65-aaeb-b162dd4aed44 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:37.739790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-281ae105-48a5-4d65-aaeb-b162dd4aed44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.740493 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.751656 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3480/7038] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:37.753149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b43-d46d-47b0-b537-af42a0054571 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.753605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b43-d46d-47b0-b537-af42a0054571 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.753894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e6b43-d46d-47b0-b537-af42a0054571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.760551 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3559/7039] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:37 2022] PUT /identity/v3/projects/04759b8d896645a08cf0eb979a5238c9/users/223e7daaa61e4799b9586e6f7f86f253/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:37.761513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.782473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-448e6b43-d46d-47b0-b537-af42a0054571 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:37.783742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-448e6b43-d46d-47b0-b537-af42a0054571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.836187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:37.861740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3481/7040] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:37.863356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.863638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.863912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.871683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0ee11-3ae6-4234-8413-6f889db4ab35 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.879121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.879753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.880068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.880488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.881216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:37.881702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.884253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.901113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] MFA Rules not processed for user `223e7daaa61e4799b9586e6f7f86f253`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:37.902679 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3560/7041] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:37.903539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.906054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:37.915232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7ecc3-5600-4021-ac01-2db48147c4d5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.923442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.924130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.924542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.924875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.925611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:37.925975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.927777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.941930 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3561/7042] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:37.942820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:37.955390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb42fcb-7db1-43b8-9fe4-5bca552471e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:37.962434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:37.963246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:37.963527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:37.963796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:37.965618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:37.968675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:37.986795 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3482/7043] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:37.991005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.004467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720fad0-0c4e-43af-a9a9-48710b83f103 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.006967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3562/7044] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.007817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.023087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:38.107350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048da84-bd5c-4dc1-9bfc-2f00208ba084 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.114131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.114799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.115143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.115494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.115789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.116537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.120165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.120700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.121101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.121456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.122202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.125262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.171136 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3563/7045] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.171136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.171747 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3483/7046] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:37 2022] POST /identity/v3/projects => generated 375 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.173482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.283568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20325e-4a3a-49b3-8051-54197d1a10eb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.305245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7396c7d-f3ee-40bc-a82a-21f357fe8852 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.322196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.323106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.323547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.324023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.324815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.330786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:38.331449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.381671 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3484/7047] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.382626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.395094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.396217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.396333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.396522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.397255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.398685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eeb2cc-3d3d-442f-b625-53949e4da7af admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.400023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.404988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.405953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.406298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.409941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.410879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.414703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.432360 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3564/7048] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/users => generated 288 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.432360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.442579 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3485/7049] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/users => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:38.442579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.443884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9baf98-a959-42a4-86f7-61653bb21649 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.452309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.453020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.454009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.454362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.455115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.455243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297b5f6-cc30-4747-85db-c0771bb87e57 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.456872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.460062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.460668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.461066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.461451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.462092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.463820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.470697 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3565/7050] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.470697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.476107 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3486/7051] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.477009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.483569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1708-9a0d-4185-850d-aca6881cadaa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.488315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.488788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eb27a-0920-4776-affe-3b4408d1d233 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.488919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.489181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.489455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.490090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.491759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.494862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.495500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.495783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.496051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] PATH_INFO: `/v3/projects/c1e1703353824b92b74e98c6f731f733/users/e04c8f174bb44741bc0e681ff15141b8/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.506605 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3566/7052] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.506605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.516420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd61d18-60b0-42e4-b18d-54a9652cfea9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.517865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e1703353824b92b74e98c6f731f733, user_id=e04c8f174bb44741bc0e681ff15141b8, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.520836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.522614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.523245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.524681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.524925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.525646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.527647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.541996 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3567/7053] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.543028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.550979 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3487/7054] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:38 2022] PUT /identity/v3/projects/c1e1703353824b92b74e98c6f731f733/users/e04c8f174bb44741bc0e681ff15141b8/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:38.554885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.558570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e809-6ebf-4f5c-9051-c31eac2cee27 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.606833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:38.631337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.631981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.632094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045fd0-ba47-4c89-981c-5fb93d24506a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.632384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.632817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.634329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.636522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.638893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.639573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.639990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.640271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] PATH_INFO: `/v3/projects/0226252ba9db43729f7d3ee01f771f69/users/221a4caf0657492a9e7cdaf8b4ee3084/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.649861 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3568/7055] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.650755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.658274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0226252ba9db43729f7d3ee01f771f69, user_id=221a4caf0657492a9e7cdaf8b4ee3084, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.661253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.661868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b86802-f3da-4677-8216-b31ae7551c21 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.667776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.668617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.669012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.669429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.670365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.673175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.676527 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3488/7056] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:38 2022] PUT /identity/v3/projects/0226252ba9db43729f7d3ee01f771f69/users/221a4caf0657492a9e7cdaf8b4ee3084/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:38.683471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.690562 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3569/7057] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:38.693668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:38.710108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dacdf8-e5e1-4dc6-9b1d-b4da35cc510f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.807040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ac7c8-fa90-40d4-ab36-471014c85e9b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:38.812066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.812735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.813023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.813291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] PATH_INFO: `/v3/projects/6ecb9284c1e149c7a05e6065bf8762a9/users/992bb56657f340919192b7624025e253/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.824170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:38.824957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.825304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.825686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] PATH_INFO: `/v3/projects/c1e1703353824b92b74e98c6f731f733/users/e04c8f174bb44741bc0e681ff15141b8/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.828573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e1703353824b92b74e98c6f731f733, user_id=e04c8f174bb44741bc0e681ff15141b8, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.830285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecb9284c1e149c7a05e6065bf8762a9, user_id=992bb56657f340919192b7624025e253, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:38.833404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b0e77-7c45-4e15-87d6-5e6794acc4ca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.833555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72cca2-0954-43bc-9af1-11057bb0c059 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:38.850574 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3570/7058] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:38 2022] PUT /identity/v3/projects/c1e1703353824b92b74e98c6f731f733/users/e04c8f174bb44741bc0e681ff15141b8/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:38.856012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.856625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.857281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.863882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3489/7059] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:38 2022] PUT /identity/v3/projects/6ecb9284c1e149c7a05e6065bf8762a9/users/992bb56657f340919192b7624025e253/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:38.865195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada39444-aaac-4a6b-8e80-c82422429d49 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.865484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada39444-aaac-4a6b-8e80-c82422429d49 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.865763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada39444-aaac-4a6b-8e80-c82422429d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.880131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:38.897868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ada39444-aaac-4a6b-8e80-c82422429d49 None None] MFA Rules not processed for user `e04c8f174bb44741bc0e681ff15141b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:38.899339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ada39444-aaac-4a6b-8e80-c82422429d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:38.902314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] MFA Rules not processed for user `221a4caf0657492a9e7cdaf8b4ee3084`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:38.904446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:38.983213 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3490/7060] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:38.983213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:38.983213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:38.983213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:38.998197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3571/7061] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:38.999203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:39.000752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:39.010648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:39.012366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:39.043898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:39.087988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3491/7062] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:39.088993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:39.090370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:39.098262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269579e4-f44c-4fbf-b3e0-6fe063bb94c7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:39.102602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:39.103260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:39.105284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:39.105284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:39.110877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a50345-3587-443f-a2ec-b74106b36749 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:39.115768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:39.116377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:39.116657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:39.116929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:39.128220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:39.130128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:39.141449 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3492/7063] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:39.142195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:39.147478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:39.175503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:39.177836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:39.192124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3572/7064] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:39.248035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276848-7562-48fd-b844-52d1f7aeb7bd service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:39.253668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:39.254423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:39.254935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:39.255476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:39.270477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:39.272385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:39.287150 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3493/7065] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:41.117788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:41.127419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:41.135121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be147-a1f4-4114-a3bf-0602ffa1e571 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:41.145397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:41.146176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.146472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.146769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] PATH_INFO: `/v3/users/221a4caf0657492a9e7cdaf8b4ee3084` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:41.147800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9578b10-d5e2-4079-b40e-48d4572982ef service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:41.152565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221a4caf0657492a9e7cdaf8b4ee3084)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:41.158048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:41.162555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:41.165487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.165810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.166089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] PATH_INFO: `/v3/users/223e7daaa61e4799b9586e6f7f86f253` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:41.170404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] RBAC: Authorizing `identity:delete_user(user_id=223e7daaa61e4799b9586e6f7f86f253)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:41.176144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:41.280268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:41.285185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221a4caf0657492a9e7cdaf8b4ee3084'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.287533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221a4caf0657492a9e7cdaf8b4ee3084'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.305105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221a4caf0657492a9e7cdaf8b4ee3084'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.314806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223e7daaa61e4799b9586e6f7f86f253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.316079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223e7daaa61e4799b9586e6f7f86f253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.327185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221a4caf0657492a9e7cdaf8b4ee3084'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.328016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-eb8378a6-8ec8-42f1-aca3-37fab29b3326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221a4caf0657492a9e7cdaf8b4ee3084'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.333614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223e7daaa61e4799b9586e6f7f86f253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.353533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3573/7066] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:41 2022] DELETE /identity/v3/users/221a4caf0657492a9e7cdaf8b4ee3084 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:41.359007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.359300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223e7daaa61e4799b9586e6f7f86f253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.359527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.359984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:41.360605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bc669b7d-f379-4ffd-a6d7-70cb132b886c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223e7daaa61e4799b9586e6f7f86f253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:41.415202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:41.419579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:41.431798 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3494/7067] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:41 2022] DELETE /identity/v3/users/223e7daaa61e4799b9586e6f7f86f253 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:41.451450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.451730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.452011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:41.506408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:41.507750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:41.575218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:41.584573 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3574/7068] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:41.653959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3495/7069] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:41.660177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:41.662037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:41.687084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:41.688996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:41.796545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab7b50-4882-4479-a4c5-7b7dc22ad482 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:41.841427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c00b08-27e8-4d14-a1f6-f3b17f450d90 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:41.851063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:41.852934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.853379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.853816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:41.877807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:41.879727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:41.904020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3496/7070] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:41.955827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:41.960297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:41.960598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:41.961570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:42.000715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:42.003964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:42.034281 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3575/7071] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:43.223806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:43.225379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:43.258399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07b389-3634-44b2-9d3e-bc1fb66e70a4 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:43.276674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:43.279731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:43.285280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:43.285874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] PATH_INFO: `/v3/projects/04759b8d896645a08cf0eb979a5238c9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:43.287757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04759b8d896645a08cf0eb979a5238c9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:43.291543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:43.412056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04759b8d896645a08cf0eb979a5238c9', 'request_id': 'req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.423858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04759b8d896645a08cf0eb979a5238c9', 'request_id': 'req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.448821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04759b8d896645a08cf0eb979a5238c9', 'request_id': 'req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.469143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] The token cache is being invalidate because project 04759b8d896645a08cf0eb979a5238c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:43.469715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.470883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.477237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3497/7072] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:43 2022] DELETE /identity/v3/projects/04759b8d896645a08cf0eb979a5238c9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:43.478344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:43.504993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf7d3c-fddd-4dc7-9965-23e0c4298b9a service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:43.548375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:43.621413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:43.622429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:43.622986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:43.623445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] PATH_INFO: `/v3/users/e04c8f174bb44741bc0e681ff15141b8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:43.625947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e04c8f174bb44741bc0e681ff15141b8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:43.630091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:43.751771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e04c8f174bb44741bc0e681ff15141b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.752871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e04c8f174bb44741bc0e681ff15141b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.765483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e04c8f174bb44741bc0e681ff15141b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.785554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e04c8f174bb44741bc0e681ff15141b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.786576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b59a0c9-ea60-4515-bd3e-d08061f45f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e04c8f174bb44741bc0e681ff15141b8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:43.807716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:43.809223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:43.813740 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3576/7073] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:43 2022] DELETE /identity/v3/users/e04c8f174bb44741bc0e681ff15141b8 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:43.842722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8fd79-7f54-4d63-9b91-cea03c62fed8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:43.952327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:43.960672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:43.962335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:43.962971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:43.963502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] PATH_INFO: `/v3/projects/0226252ba9db43729f7d3ee01f771f69` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:43.965148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0226252ba9db43729f7d3ee01f771f69)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:43.967638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:44.079079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0226252ba9db43729f7d3ee01f771f69', 'request_id': 'req-a6605018-a269-4772-b1b2-c3970b067138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:44.091614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0226252ba9db43729f7d3ee01f771f69', 'request_id': 'req-a6605018-a269-4772-b1b2-c3970b067138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:44.112822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0226252ba9db43729f7d3ee01f771f69', 'request_id': 'req-a6605018-a269-4772-b1b2-c3970b067138'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:44.137147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] The token cache is being invalidate because project 0226252ba9db43729f7d3ee01f771f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:44.137784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:44.138927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6605018-a269-4772-b1b2-c3970b067138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:44.143310 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3498/7074] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:43 2022] DELETE /identity/v3/projects/0226252ba9db43729f7d3ee01f771f69 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:44.149868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:44.151126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:44.152613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:44.184145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:44.185365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:44.294375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3577/7075] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:44.301057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:44.301596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:44.301596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:44.343769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:44.346627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:44.366641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3499/7076] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:44.371291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:44.394081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405580b9-b663-4c6c-8d38-75c227ee1ca2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:44.408387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:44.411967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:44.412148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:44.421978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:44.424113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:44.426358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:44.434086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:44.475317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3578/7077] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:44.479818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:44.502676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c305d9-3775-4df3-93ef-c52157add133 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:44.510279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:44.510279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:44.510279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:44.510279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:44.510728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:44.511564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:44.573823 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3500/7078] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:44 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:44.579488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:44.616527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366918be-f4d7-485f-a823-643a39e84843 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:44.747335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:44.870340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:44.873411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:44.874333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:44.875157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:44.877101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:44.893184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:44.974845 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3579/7079] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:44 2022] POST /identity/v3/users => generated 283 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:44.975888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:44.993508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ff5a9-2862-49df-866d-40c1bbe8cfed admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:45.004588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:45.005518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.005802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.006078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.006759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:45.017836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:45.088068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:45.091366 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3501/7080] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:45.118007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a404f75-6365-4c39-b8b2-857cf54e0dde admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:45.118305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:45.122853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:45.138398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:45.139599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.140857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.142358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.144228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:45.148322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:45.162231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae70988-52b8-4ee1-a06e-8d5a001796b3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:45.176769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:45.177429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.180808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.181198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] PATH_INFO: `/v3/projects/c1e1703353824b92b74e98c6f731f733` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.182986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e1703353824b92b74e98c6f731f733)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:45.186626 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3580/7081] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:45.187286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:45.190437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:45.220462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb81323-fa32-40a9-9664-99f4b990dddc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:45.235372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:45.237048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.237553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.238044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] PATH_INFO: `/v3/projects/664dbade7a4c4d59aedb6fb3d7dc1926/users/7d8dac4f77b644c890dc6f25ca82276e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.330208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=664dbade7a4c4d59aedb6fb3d7dc1926, user_id=7d8dac4f77b644c890dc6f25ca82276e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:45.330422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e1703353824b92b74e98c6f731f733', 'request_id': 'req-96cbb4d0-fed5-484c-bc40-f30662210228'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:45.339242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c1daa-5ca4-460a-96dd-389f85c169b6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:45.339962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e1703353824b92b74e98c6f731f733', 'request_id': 'req-96cbb4d0-fed5-484c-bc40-f30662210228'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:45.357807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e1703353824b92b74e98c6f731f733', 'request_id': 'req-96cbb4d0-fed5-484c-bc40-f30662210228'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:45.396517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3581/7082] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:45 2022] PUT /identity/v3/projects/664dbade7a4c4d59aedb6fb3d7dc1926/users/7d8dac4f77b644c890dc6f25ca82276e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:45.399558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.400056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.400398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.405856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] The token cache is being invalidate because project c1e1703353824b92b74e98c6f731f733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:45.406365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:45.407419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-96cbb4d0-fed5-484c-bc40-f30662210228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:45.410646 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3502/7083] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:45 2022] DELETE /identity/v3/projects/c1e1703353824b92b74e98c6f731f733 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:15:45.427684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e265-8eea-4221-89ec-d70861c1921e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.429617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e265-8eea-4221-89ec-d70861c1921e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.430790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1e265-8eea-4221-89ec-d70861c1921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.535492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-49c1e265-8eea-4221-89ec-d70861c1921e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:45.537060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-49c1e265-8eea-4221-89ec-d70861c1921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:45.572152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] MFA Rules not processed for user `7d8dac4f77b644c890dc6f25ca82276e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:45.578360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:45.680325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:45.788254 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3503/7084] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:45.813239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.813717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.814241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.891735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:45.892983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:45.906997 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3582/7085] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:45.919827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:45.921702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:45.925290 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3504/7086] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:45.954583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:45.966122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34316d-180c-43c6-b503-4d292d7ed1f5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:45.978792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:45.979672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:45.980103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:45.980599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:45.981512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:45.982022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:45.983912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.016769 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3505/7087] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:46.018442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.044954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a32f-0526-4401-b0c8-a62f72381331 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.050096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.050880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.051260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.051639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.052364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.054399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.102756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573111-1820-4ec3-9235-4be7403de49d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.115273 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3506/7088] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:46 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:46.119141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.119280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.120032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.120323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.120595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.141750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.145133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.174967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3583/7089] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:46.211783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219aa7-0d43-42a8-84fc-c8b52f1ced56 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.217487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.218365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.218687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.218963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.219883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.223793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.258284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3507/7090] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:46 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:46.263024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.292823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ae923-8b69-4e99-bb64-ecb5ed38af3a service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.316359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.317711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.318293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.318770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.319637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.326305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.347200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3584/7091] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:15:46.351974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.368354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25327fa-c396-4f55-bd69-f2af7afd3b02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.381010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.385499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.387353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.389143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.389972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.392392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.426080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3508/7092] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:46.430695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.441973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:46.459928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bf503-823d-4f33-9148-24bb3bde8007 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.473057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.475005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.475802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.476181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.476361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:46.479368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] PATH_INFO: `/v3/projects/a5c4c9ce4cb946a9a381271ab15aee09/users/82c2160fcae947138ddd7b1b4df1e361/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.537729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c4c9ce4cb946a9a381271ab15aee09, user_id=82c2160fcae947138ddd7b1b4df1e361, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.547166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c25fef7-a1cd-4090-843d-ce6223db06e6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.580108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3585/7093] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:46 2022] PUT /identity/v3/projects/a5c4c9ce4cb946a9a381271ab15aee09/users/82c2160fcae947138ddd7b1b4df1e361/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:46.593760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.594457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.595455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.660677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c649-9633-4cea-bea8-58e27fabc498 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.680963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] MFA Rules not processed for user `82c2160fcae947138ddd7b1b4df1e361`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:46.682254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:46.728802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:46.839566 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3586/7094] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:46.841647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:46.872015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.873722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.874302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.874302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.909551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:46.936319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.937866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:46.953087 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3509/7095] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:46.977414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbc39e-b000-4aca-8631-75f271a8662c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:46.983700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:46.984443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:46.984850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:46.985254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:46.985984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:46.988184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.060871 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3587/7096] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:46 2022] POST /identity/v3/projects => generated 393 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:47.062889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.080930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69f0a3-421f-425d-b03e-7fe9d5de2e23 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:47.168002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:47.168730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.169086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.169436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.170482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:47.172545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.207767 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3510/7097] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:47 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:47.209334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.224229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22096142-32ce-4a86-af11-b86f7689dd07 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:47.232425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:47.233437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.234070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.234634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.235563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:47.237780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.254911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3588/7098] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:47.255963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.270328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01bb-e59c-4759-b51c-60b856eb20e7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:47.277147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:47.277826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.278104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.278385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] PATH_INFO: `/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd/users/3246fd815aa74d33b8eaeffcd719e2e1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.301082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f4287f4d0b4f2281d6c40edcba11fd, user_id=3246fd815aa74d33b8eaeffcd719e2e1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:47.306037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.328992 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3511/7099] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:47 2022] PUT /identity/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd/users/3246fd815aa74d33b8eaeffcd719e2e1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:47.330870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.346705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a703-29d0-4211-8ca3-49992ac7f049 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:47.447284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:47.488378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:47.489900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.490399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.491096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.491956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:47.497289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b9883-6f10-44d2-b03a-5998f1581d63 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.515696 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3589/7100] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:47.516881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.536507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8bce-d5c4-406e-875d-1ebef2de0710 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:47.542317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:47.543431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.544421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.544802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] PATH_INFO: `/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd/users/3246fd815aa74d33b8eaeffcd719e2e1/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.547389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f4287f4d0b4f2281d6c40edcba11fd, user_id=3246fd815aa74d33b8eaeffcd719e2e1, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:47.550419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:47.583066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:47.583637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:47.583959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:47.587612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3512/7101] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:47 2022] PUT /identity/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd/users/3246fd815aa74d33b8eaeffcd719e2e1/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:47.625870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] MFA Rules not processed for user `3246fd815aa74d33b8eaeffcd719e2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:47.627523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:47.763802 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3590/7102] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:47.772662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:47.778388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:47.996225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7642-b7f3-4f12-af30-e3428b4dfcec admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:48.018466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:48.019711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:48.019988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:48.020258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:48.060459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:48.219717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:48.221543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38bc2b-5ed7-4306-ba7a-09f8a07970d9 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:48.241331 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3513/7103] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:53.118977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:53.121687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:53.215713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:53.216258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:53.216258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:53.329659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] MFA Rules not processed for user `992bb56657f340919192b7624025e253`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:53.330944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:53.339524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16505-31be-4b44-87ef-eb7358ad0bbb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:53.347612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:53.349512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:53.350744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:53.351398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] PATH_INFO: `/v3/users/7d8dac4f77b644c890dc6f25ca82276e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:53.353521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8dac4f77b644c890dc6f25ca82276e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:53.355678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:53.437198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8dac4f77b644c890dc6f25ca82276e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:53.438088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8dac4f77b644c890dc6f25ca82276e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:53.450939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8dac4f77b644c890dc6f25ca82276e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:53.484222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:53.488556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8dac4f77b644c890dc6f25ca82276e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:53.489366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-47753690-ff71-4811-a6b7-b9d40a157780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8dac4f77b644c890dc6f25ca82276e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:53.517008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3591/7104] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:53 2022] DELETE /identity/v3/users/7d8dac4f77b644c890dc6f25ca82276e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:53.531118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:53.533247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:53.537372 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3514/7105] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:53.537536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:53.539826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:53.543319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:53.569053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:53.571642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:53.578686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8af97-49db-467f-80aa-9320350f2e51 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:53.707409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:53.707955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:53.708247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:53.708521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:53.709162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:53.713831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:53.784283 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3592/7106] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:53.791819 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3515/7107] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:53 2022] POST /identity/v3/projects => generated 375 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:53.794091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:53.812551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc1a6f-1648-4898-bd73-e444443e8882 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:53.842088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:53.844717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:53.948898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:54.025068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.025756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.026031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.026303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.030275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.030890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774abfd5-c3a7-4c8b-8507-a2cbe614f925 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:54.035841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28526d98-648e-4ef6-bfd1-221e9cd961ab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.044352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.044990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.045276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.045956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.074547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.100939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.106444 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3593/7108] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:53 2022] POST /identity/v3/users => generated 294 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:54.115282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.115605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.117477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.171062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3516/7109] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:54.174522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:54.197209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] MFA Rules not processed for user `c1acf8d4b1384c8c8d7014bd8baf897e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:54.198528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:54.203041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9f073-d05c-41d0-9166-881ba7e6b025 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:54.216301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.218161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.218566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.221359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.224245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.226278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.261904 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3517/7110] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:54.267508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:54.299523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24559a-df5a-4bd4-b472-638a64ef613b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:54.318002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.319467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.319467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.319467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] PATH_INFO: `/v3/projects/0c0033b625ca4995bff138c485673fd8/users/fc712fcf0f814a59bb52e16685ab1c9b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.333971 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3594/7111] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:54.402729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0033b625ca4995bff138c485673fd8, user_id=fc712fcf0f814a59bb52e16685ab1c9b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.410414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.443870 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3518/7112] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:54 2022] PUT /identity/v3/projects/0c0033b625ca4995bff138c485673fd8/users/fc712fcf0f814a59bb52e16685ab1c9b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:54.448515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:54.467143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f589ab-0364-42dd-956a-24fa2038ea34 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:54.524452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:54.654891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.655840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.656328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.656677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.657831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.660631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.703438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3595/7113] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:54.705499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:54.750688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc9cf-f94d-4721-93cb-8141e1d3bc2b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:54.762298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:54.764787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:54.765882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:54.766049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] PATH_INFO: `/v3/projects/0c0033b625ca4995bff138c485673fd8/users/fc712fcf0f814a59bb52e16685ab1c9b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:54.772024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0033b625ca4995bff138c485673fd8, user_id=fc712fcf0f814a59bb52e16685ab1c9b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:54.782710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb688b52-2f5f-4376-922c-4ee6068ba505 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:54.813229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3519/7114] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:54 2022] PUT /identity/v3/projects/0c0033b625ca4995bff138c485673fd8/users/fc712fcf0f814a59bb52e16685ab1c9b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:54.942796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:54.945311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:54.968394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c9c0-fc87-4e1b-8fdd-e3db79a1cadb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:55.082689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:55.083349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.083626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.083942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] PATH_INFO: `/v3/projects/664dbade7a4c4d59aedb6fb3d7dc1926` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.086278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664dbade7a4c4d59aedb6fb3d7dc1926)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:55.090097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:55.178543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664dbade7a4c4d59aedb6fb3d7dc1926', 'request_id': 'req-723e9f9d-e214-4b8f-b4cd-d35ba075e514'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:55.188822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664dbade7a4c4d59aedb6fb3d7dc1926', 'request_id': 'req-723e9f9d-e214-4b8f-b4cd-d35ba075e514'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:55.199110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664dbade7a4c4d59aedb6fb3d7dc1926', 'request_id': 'req-723e9f9d-e214-4b8f-b4cd-d35ba075e514'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:55.219716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] The token cache is being invalidate because project 664dbade7a4c4d59aedb6fb3d7dc1926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:55.220435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:55.221276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-723e9f9d-e214-4b8f-b4cd-d35ba075e514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:55.232282 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3596/7115] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:54 2022] DELETE /identity/v3/projects/664dbade7a4c4d59aedb6fb3d7dc1926 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:55.237105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.237635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.237635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.275948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:55.279169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:55.439425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3520/7116] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:55.449918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.450205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.450949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.549485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:55.551049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:55.570607 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3597/7117] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:55.573803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:55.588755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e0bb4-7e84-4a6b-9095-2f602af243f9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:55.595442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:55.596341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.596876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.597299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.598323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:55.598431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:55.603073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:55.638436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:55.640168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3521/7118] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:15:55.649670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e17d7-b2a7-4e2d-a59a-209f11b13df2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:55.655457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:55.656336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.656991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.657425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.658231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:55.660369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:55.717486 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3598/7119] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:55.721937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:55.756426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd533-c7d0-4f4c-9e2d-9613de3a93f1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:55.906125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:55.907065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:55.907520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:55.908501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:55.909333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:55.911585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:55.957684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3522/7120] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:55 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:55.964009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:55.980465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f8f0a-4ba8-42bc-9ad8-9bc8da63bb16 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:55.999244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:56.000141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.000585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.001019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.002617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:56.004332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:56.028035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:56.033979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:56.034369 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3599/7121] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:56.067126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b2466-1ff7-44cc-a490-cb98099a98ae admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:56.093730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:56.094365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.094728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.095008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.095657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:56.097273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6b720-7b0f-4c5f-aa54-414a874f9d73 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:56.147543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:56.156867 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3523/7122] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:15:56.169630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063da8ff-9dc0-488c-ac3a-7c700bb6e4c1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:56.184620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:56.185310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.185580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.185852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] PATH_INFO: `/v3/projects/de650167c2244e41908c194055d36b8e/users/d4abbc74d7fb42e484fd571b9fabd12c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.191640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:56.220589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de650167c2244e41908c194055d36b8e, user_id=d4abbc74d7fb42e484fd571b9fabd12c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:56.233654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:56.300793 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3600/7123] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:15:56 2022] PUT /identity/v3/projects/de650167c2244e41908c194055d36b8e/users/d4abbc74d7fb42e484fd571b9fabd12c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:56.304516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.307201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.307911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.398456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] MFA Rules not processed for user `d4abbc74d7fb42e484fd571b9fabd12c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:56.400272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:56.439588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:56.442058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:56.592469 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3524/7124] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:56.598971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:56.603394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:56.653325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d35372-e34d-4ec5-9764-d716f3ccbb15 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:56.672960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:56.673583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.673864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.674148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.683639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:56.703817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:56.705439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:56.738191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3601/7125] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:15:56.760060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e3f5f-92d0-4d0b-9db7-82863d4a3653 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:56.770148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:56.771049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:56.775906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:56.776354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:56.794230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:56.796206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:56.839940 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3525/7126] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:15:57.756055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:57.760046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:57.929088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5bed3-1512-4424-a372-4309de1e1e56 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:57.935273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:57.936327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:57.937780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:57.938395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] PATH_INFO: `/v3/users/57c163c13444454c9a23703c08a9cc5a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:57.939905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c163c13444454c9a23703c08a9cc5a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:57.946933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:58.081120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c163c13444454c9a23703c08a9cc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.081692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c163c13444454c9a23703c08a9cc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.100062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c163c13444454c9a23703c08a9cc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.125163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c163c13444454c9a23703c08a9cc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.126055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c163c13444454c9a23703c08a9cc5a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.152498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:58.191322 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3602/7127] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:57 2022] DELETE /identity/v3/users/57c163c13444454c9a23703c08a9cc5a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:58.599600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:58.622491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964003f9-5f22-4cc6-84c4-da02ca2868d4 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:58.820387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:58.821134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:58.821570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:58.822056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] PATH_INFO: `/v3/projects/8590b56a0c924f76ba18920e735e6b4d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:58.848366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8590b56a0c924f76ba18920e735e6b4d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:58.850265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:58.915157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:58.918415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:58.919112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:58.945942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a0bdc-2875-4bc1-a062-b08ae1a4c929 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:58.948768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8590b56a0c924f76ba18920e735e6b4d', 'request_id': 'req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.971442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8590b56a0c924f76ba18920e735e6b4d', 'request_id': 'req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:58.987045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8590b56a0c924f76ba18920e735e6b4d', 'request_id': 'req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.004773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] The token cache is being invalidate because project 8590b56a0c924f76ba18920e735e6b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:15:59.006219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.008586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e78ddc9c-8bd5-478c-a678-ffdf4edb907e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.012047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3526/7128] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:15:58 2022] DELETE /identity/v3/projects/8590b56a0c924f76ba18920e735e6b4d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:59.028697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df669-5acd-4168-b102-2ee94c876f41 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.028974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df669-5acd-4168-b102-2ee94c876f41 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.029256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099df669-5acd-4168-b102-2ee94c876f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.122815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-099df669-5acd-4168-b102-2ee94c876f41 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:59.124072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-099df669-5acd-4168-b102-2ee94c876f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:59.125754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:59.126413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.126714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.126983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] PATH_INFO: `/v3/users/d4abbc74d7fb42e484fd571b9fabd12c` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.131635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4abbc74d7fb42e484fd571b9fabd12c)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:59.138002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:59.150956 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3527/7129] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:59.161015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.161347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.161633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.218875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:59.224235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:59.245172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3528/7130] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:15:59.252420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:59.257884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4abbc74d7fb42e484fd571b9fabd12c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.258718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4abbc74d7fb42e484fd571b9fabd12c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.262445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fdcb8-884b-4a0a-a0bb-737e3ec19d94 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:59.274939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4abbc74d7fb42e484fd571b9fabd12c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.279622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:59.280248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.280633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.280918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.281570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:15:59.281869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:59.286661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:59.301875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4abbc74d7fb42e484fd571b9fabd12c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.303578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c13909ee-1c1c-4194-9272-acc1e26739f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4abbc74d7fb42e484fd571b9fabd12c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:15:59.314999 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3529/7131] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:15:59.317336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:59.319286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:59.339593 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3603/7132] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:15:58 2022] DELETE /identity/v3/users/d4abbc74d7fb42e484fd571b9fabd12c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:15:59.342155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42ef18-8b84-401c-81a5-9215bad447d3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:59.357762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.358177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.358541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.416492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:15:59.418185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:59.464674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:59.465431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.465721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.466038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.466779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:59.469302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:59.576985 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3530/7133] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:15:59 2022] POST /identity/v3/projects => generated 369 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:15:59.578203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:59.633853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3604/7134] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:15:59.643858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:59.650408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:15:59.697315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:59.774446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701cff3-f489-4d4f-8182-ff74dcab3e42 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:59.791262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:59.791987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.793238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.793598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:15:59.794321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:15:59.799762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:15:59.855657 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3531/7135] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:15:59 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:15:59.856734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:15:59.870944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:15:59.880771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafac49-8934-4ef8-83b0-e750e4e3fb4c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:15:59.988870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:15:59.989693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:15:59.989979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:15:59.990254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.004531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.006226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.013148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c13e1-c525-4bd8-ba7d-3131109e81fb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.026316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:00.027696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.028055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.028403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.031106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:00.032444 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3605/7136] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:00.033404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:00.045700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d104-7115-4577-aee8-5efe7ac6f3f2 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.082949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:00.084463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.084748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.085020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.085773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.087650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.105555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3606/7137] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:00.109564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:00.131523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f355b2-a4de-46e6-a1a8-d6f63181a0e7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.145172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:00.146122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.146418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.146776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.147551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.149792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.172190 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3607/7138] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:00.173132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:00.183842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.186299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04f31-1347-4e68-b400-a37c54382115 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.199951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9babd-285e-417c-b1e7-bf513ec3bc22 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.211978 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3532/7139] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:00.234430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58101031-e57c-413e-a642-acab032b636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:00.235394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101031-e57c-413e-a642-acab032b636c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.235810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101031-e57c-413e-a642-acab032b636c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.236214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58101031-e57c-413e-a642-acab032b636c admin admin] PATH_INFO: `/v3/projects/c22cf4343bb04e44afe9664484ecbf02/users/39a60695d08a4ae79d776a4ccdf4b348/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.265076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58101031-e57c-413e-a642-acab032b636c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22cf4343bb04e44afe9664484ecbf02, user_id=39a60695d08a4ae79d776a4ccdf4b348, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.269291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58101031-e57c-413e-a642-acab032b636c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.301556 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3608/7140] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:00 2022] PUT /identity/v3/projects/c22cf4343bb04e44afe9664484ecbf02/users/39a60695d08a4ae79d776a4ccdf4b348/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:00.312297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.312952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.316190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.368016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] MFA Rules not processed for user `39a60695d08a4ae79d776a4ccdf4b348`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:00.371044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:00.547535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3533/7141] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:00.551856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58101031-e57c-413e-a642-acab032b636c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:00.773883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58101031-e57c-413e-a642-acab032b636c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.795511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:00.796190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:00.796530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:00.796731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:00.797453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:00.799632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:00.870213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3609/7142] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:00 2022] POST /identity/v3/projects => generated 369 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:00.874267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:00.896049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff01347-b357-46dd-972e-d718a806a1f8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:00.994854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:01.040536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.041179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.041461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.041735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.042380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.044394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.125340 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3534/7143] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:00 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:01.127667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.145972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e581cf2-fc1b-474e-800d-a31268aa76e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:01.155338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.156444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.156889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.157301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.158117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.159977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.174014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:01.186763 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3610/7144] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:01.191306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.222044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618cf9f-2703-45d6-97d4-152b5e234e06 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:01.234665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.235851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.236356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.236839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] PATH_INFO: `/v3/projects/56f21092159b4fe4a8753788bbe9f3e2/users/9d0eab16387740d1b6d7ee844bef1cdd/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.260003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f21092159b4fe4a8753788bbe9f3e2, user_id=9d0eab16387740d1b6d7ee844bef1cdd, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.263489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.284519 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3535/7145] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:01 2022] PUT /identity/v3/projects/56f21092159b4fe4a8753788bbe9f3e2/users/9d0eab16387740d1b6d7ee844bef1cdd/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:01.288661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.307773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142237c2-6202-44cb-a1b3-ec9f565f25b2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:01.419168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.420151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.420661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.421218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.422062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.424105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52aec4-1a0f-4155-9221-b19460acacfb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.440137 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3611/7146] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:01.444086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.455467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7143e2-11c5-4136-a6b3-7b5ad0dda196 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:01.461884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.462937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.463571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.464056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] PATH_INFO: `/v3/projects/56f21092159b4fe4a8753788bbe9f3e2/users/9d0eab16387740d1b6d7ee844bef1cdd/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.466644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f21092159b4fe4a8753788bbe9f3e2, user_id=9d0eab16387740d1b6d7ee844bef1cdd, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.469865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.499012 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3536/7147] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:01 2022] PUT /identity/v3/projects/56f21092159b4fe4a8753788bbe9f3e2/users/9d0eab16387740d1b6d7ee844bef1cdd/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:01.589340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.590385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.590385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.628174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.637036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:01.638393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] MFA Rules not processed for user `9d0eab16387740d1b6d7ee844bef1cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:01.639905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:01.812013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3612/7148] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:01.816330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:01.853794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3cf1d-cc81-4f34-b843-c81ec5c12f95 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:01.877236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:01.880037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:01.880333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:01.880611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:01.881304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:01.898666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:01.940766 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3613/7149] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:16:01 2022] POST /identity/v3/domains => generated 276 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:01.950185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.153677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0d318-3285-4d71-a5c6-601de9b975e7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.159909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.160553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.160839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.161123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.161773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.172894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.196389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:02.222007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3614/7150] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:01 2022] POST /identity/v3/groups => generated 280 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:02.226368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.258688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cb55d-f781-4842-80b2-19b9dee92678 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.262711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84892d-db4e-4cc5-89c8-1e34f400da76 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.274042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.275043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.275813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.275813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/89e22d61e78e4a819652692672a229a3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.282002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:update_group(group_id=89e22d61e78e4a819652692672a229a3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.283104 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:02.283104 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:02.283104 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:02.284365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.321230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3615/7151] 10.222.0.52 () {60 vars in 1247 bytes} [Sat Sep 24 21:16:02 2022] PATCH /identity/v3/groups/89e22d61e78e4a819652692672a229a3 => generated 291 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:02.322671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.343650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52e1cb-d3df-446b-8f38-eebcdc5266b8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.356266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.356927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.357235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.357484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/89e22d61e78e4a819652692672a229a3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.376349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:get_group(group_id=89e22d61e78e4a819652692672a229a3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.377454 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:02.377454 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:02.377454 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:02.381592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.384280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.389719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.390299 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3616/7152] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:02 2022] GET /identity/v3/groups/89e22d61e78e4a819652692672a229a3 => generated 291 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:02.390432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.391407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] PATH_INFO: `/v3/projects/de650167c2244e41908c194055d36b8e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.392977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=de650167c2244e41908c194055d36b8e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.397363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.398401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.410688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01f27-c342-4eaa-8271-11abbeb58c55 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.423886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.424732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.425029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.425938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/89e22d61e78e4a819652692672a229a3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.427545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:update_group(group_id=89e22d61e78e4a819652692672a229a3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.429903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.466867 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3617/7153] 10.222.0.52 () {60 vars in 1246 bytes} [Sat Sep 24 21:16:02 2022] PATCH /identity/v3/groups/89e22d61e78e4a819652692672a229a3 => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:02.468704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.499080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e85d-3ca9-4575-b497-52d7b18e810f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.503365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:02.508431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de650167c2244e41908c194055d36b8e', 'request_id': 'req-2749f2ba-db85-4175-91f0-70cda90e2ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:02.516779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de650167c2244e41908c194055d36b8e', 'request_id': 'req-2749f2ba-db85-4175-91f0-70cda90e2ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:02.528337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de650167c2244e41908c194055d36b8e', 'request_id': 'req-2749f2ba-db85-4175-91f0-70cda90e2ece'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:02.559166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] The token cache is being invalidate because project de650167c2244e41908c194055d36b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:02.560159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:02.561317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2749f2ba-db85-4175-91f0-70cda90e2ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:02.569698 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3537/7154] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:01 2022] DELETE /identity/v3/projects/de650167c2244e41908c194055d36b8e => generated 0 bytes in 941 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:02.583294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182d27-e272-4891-8970-094f9674494d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.585542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182d27-e272-4891-8970-094f9674494d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.587353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182d27-e272-4891-8970-094f9674494d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.617614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83182d27-e272-4891-8970-094f9674494d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:02.627537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83182d27-e272-4891-8970-094f9674494d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:02.645374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.646177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.646639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.646991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/89e22d61e78e4a819652692672a229a3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.654440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=89e22d61e78e4a819652692672a229a3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.656805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.728142 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3618/7155] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:02 2022] DELETE /identity/v3/groups/89e22d61e78e4a819652692672a229a3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:02.731689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.734059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3538/7156] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:02.735457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:02.741878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.742461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.742957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.772299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:02.774697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:02.850148 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3539/7157] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:02.854526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.865051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e02291-2f40-4bd5-9047-746d95fa83fa None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.872004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.873876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.874580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.875070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.875987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:02.876344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.878160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.906064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48ecb1-d19f-4235-bec8-e8b29ca32c8b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.909881 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3540/7158] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:02.911538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.912118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.923882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b144a-bdbe-4450-8f77-cc8920f4de26 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.924205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.924565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.924989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.926007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.930251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.930855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:02.931749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:02.932128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:02.932643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:02.933383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:02.935799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:02.936677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:UPDATE=2,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:02.963887 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3619/7159] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/groups => generated 324 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:02.966840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:02.982789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110100f-d061-4f5f-a8e2-43689ed4e0f8 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:02.987069 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3541/7160] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:02.988419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.081900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:03.083229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.085816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.086474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.089453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.091402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.096296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.128745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b89bd8-01a4-4412-a203-862a93dc2773 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.142768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.143400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.143681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.143962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.146866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.148387 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3620/7161] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/users => generated 321 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:03.155771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.159444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.183566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126adbe7-47c3-4595-979e-4528158061ac tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.202156 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3542/7162] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:02 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:03.203643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.204426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.208225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.209687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.209687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.226590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f38f-9120-4096-9910-0bbd933b4d8f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.237986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.239035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.239035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.239233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.242359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.245405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.249810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=2676a4dfc54e48eab20ae56d7e4c8bf0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.253011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.279684 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3543/7163] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:03.279684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.302542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c77df-bff6-40a0-b871-2251d9cd2272 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.315378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.315923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.316339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.316621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.317318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.321394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.339742 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3544/7164] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:03.339963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.352805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd24f-886b-44f6-a193-7df9d47f0ae1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.370840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3621/7165] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:03 2022] PUT /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:03.371812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.487232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.488216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.488653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.489618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] PATH_INFO: `/v3/projects/c5b2658604ca463fb08b152900e01798/users/99999871759d495a90a2ef21fc5c6969/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.501169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f0ca7-3ede-4097-abc2-4f1fa7c05389 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.514176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.514856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.515228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.515748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.516178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.518461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.519756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b2658604ca463fb08b152900e01798, user_id=99999871759d495a90a2ef21fc5c6969, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.544745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06dc0d6-57ed-472d-96ef-34864c8872a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.578420 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3622/7166] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:03 2022] POST /identity/v3/users => generated 323 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:03.582960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.588515 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3545/7167] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:03 2022] PUT /identity/v3/projects/c5b2658604ca463fb08b152900e01798/users/99999871759d495a90a2ef21fc5c6969/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:03.602068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.602952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.603241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.613794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff96f67-e1ab-4542-ba35-e2b3ac4dd8cc tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:03.703594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:03.712851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] MFA Rules not processed for user `99999871759d495a90a2ef21fc5c6969`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:03.718432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:03.755856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:03.756537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:03.756878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:03.757148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:03.790714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=1b0f367c561349f3a6e50b5ef79bd03d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:03.794881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:03.869200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3623/7168] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:03 2022] PUT /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:03.870436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.919944 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3546/7169] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:03.923027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:03.967640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:04.028036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bed4-52b2-4564-be02-b7b9b4813434 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.052950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.055384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.055675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.056009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.056670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.059266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.107160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b9146-30d2-4e04-9d59-0f42f31b1f61 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.140567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3624/7170] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:03 2022] POST /identity/v3/users => generated 323 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:04.145644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.180784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cead86-fbe6-4e1d-9ffb-be7b1b1d2eb0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.193418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.196210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.197149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.197710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.250666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.251301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.251586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.251864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.252503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.262304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.269619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=37ba8132af8041fd9f8a1f28238e1802)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.297174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.324087 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3547/7171] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:04.328481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.360242 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3625/7172] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:04 2022] PUT /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:04.364313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.501182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2133009-7aa1-4e7e-9a32-54314fc5c1b7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.502467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c031b-3d48-4a32-bc44-0f1d84e11c58 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.516416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.517318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.517318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.517627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.519243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=3986bda9c145457bb1b11dc192b8ca68)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.520274 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:04.520274 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:04.520274 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:04.521476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.565276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3626/7173] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:16:04 2022] GET /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users => generated 1083 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:04.569898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.578798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:04.594431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ec890-d661-484a-b7f1-7e223bc1784c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.608068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.608747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.609024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.609301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.619926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=2676a4dfc54e48eab20ae56d7e4c8bf0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.621093 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:04.621093 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:04.621093 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:04.622432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.630096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:04.640295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.641073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.641389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.641740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.642864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.645767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.674680 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3548/7174] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:04 2022] POST /identity/v3/users => generated 301 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:04.674813 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3627/7175] 10.222.0.52 () {58 vars in 1342 bytes} [Sat Sep 24 21:16:04 2022] HEAD /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:04.678192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.680931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.702919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6c0bf-98aa-4e92-8941-549bb31ce3be tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.713336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd80e32-9820-44ac-b33b-fbfc1c4a72ca admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.716968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.718914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.719335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.719791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.723082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=2676a4dfc54e48eab20ae56d7e4c8bf0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.726355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.732896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.733669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.734802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.735185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.735942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.738389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.756269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:04.774097 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3549/7176] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:04.791073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.804719 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3628/7177] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:04 2022] DELETE /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/2676a4dfc54e48eab20ae56d7e4c8bf0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:04.812951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:04.824826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1eb78-393c-4e6c-aa3d-beeb5aa042c4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:04.923877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:04.923877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:04.923877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:04.924972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] PATH_INFO: `/v3/projects/392f324f6e7f44bbab592758663edb82/users/947800417f6d4a20ae2d27903dd6a5c7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:04.946558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392f324f6e7f44bbab592758663edb82, user_id=947800417f6d4a20ae2d27903dd6a5c7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:04.953310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14128ff0-9182-4833-b837-2e303b172a88 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:04.989594 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3550/7178] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:04 2022] PUT /identity/v3/projects/392f324f6e7f44bbab592758663edb82/users/947800417f6d4a20ae2d27903dd6a5c7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:05.002176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.002471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.002775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.025190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c57f4-f1e9-400c-ab34-4cfa92253721 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:05.068907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] MFA Rules not processed for user `947800417f6d4a20ae2d27903dd6a5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:05.073066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:05.169904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:05.170699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.171052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.171340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.176706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=1b0f367c561349f3a6e50b5ef79bd03d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:05.180301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:05.232667 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3629/7179] 10.222.0.52 () {58 vars in 1342 bytes} [Sat Sep 24 21:16:04 2022] HEAD /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:05.235775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:05.246921 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3551/7180] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3271 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:05.319722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94d621-a573-4ec2-9d94-812aa08ce80c tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:05.333004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:05.337183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:05.359907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:05.360589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.360876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.361156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.366037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=1b0f367c561349f3a6e50b5ef79bd03d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:05.372409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:05.397347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:05.471608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3630/7181] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:05 2022] DELETE /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/1b0f367c561349f3a6e50b5ef79bd03d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:05.479797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:05.483441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:05.579728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7590b3f-5974-43c4-8b12-c068852a1c3b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:05.670978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:05.767091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43908882-b715-4156-97c5-9192f1a5aad9 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:05.780812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:05.781739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.781861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:05.782398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.782718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.783082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.783198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.783463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.789251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=37ba8132af8041fd9f8a1f28238e1802)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:05.791361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:05.825722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3631/7182] 10.222.0.52 () {58 vars in 1342 bytes} [Sat Sep 24 21:16:05 2022] HEAD /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:05.826528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:05.830213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:05.925067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:05.926809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:05.947308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3552/7183] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:16:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 680 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:16:05.951121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:05.978170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95bde-fbd9-4d9e-9ba2-d63ab48397ef service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:05.988228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:05.989268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:05.990380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:05.990779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:05.995065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=3986bda9c145457bb1b11dc192b8ca68, user_id=37ba8132af8041fd9f8a1f28238e1802)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:06.011536 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:06.011536 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:06.011536 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:06.015669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:06.022108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efef050-99bd-471a-a76f-50da57eacf1a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:06.038203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:06.039162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:06.039456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:06.039774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:06.041990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:06.042251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.072230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:06.073853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:06.074329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3553/7184] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:05 2022] DELETE /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users/37ba8132af8041fd9f8a1f28238e1802 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:06.081280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:06.107305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3632/7185] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:06.266019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8fdd2-6af7-43a6-9c61-9dd19e3f6697 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:06.281928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:06.284062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:06.284443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:06.284861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:06.287342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=3986bda9c145457bb1b11dc192b8ca68)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:06.293878 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:06.293878 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:06.293878 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:06.298439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:06.326782 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3554/7186] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:16:06 2022] GET /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68/users => generated 145 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:06.329967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:06.354135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e56c50-0d30-4583-8839-bc19a389dc34 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:06.359234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:06.359882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:06.360197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:06.360479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users/37ba8132af8041fd9f8a1f28238e1802` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:06.367741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_user(user_id=37ba8132af8041fd9f8a1f28238e1802)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:06.369719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:06.494018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.495741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.518748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.554660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.557437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ba8132af8041fd9f8a1f28238e1802'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:06.605704 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3633/7187] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:06 2022] DELETE /identity/v3/users/37ba8132af8041fd9f8a1f28238e1802 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:06.610446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:06.613261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:06.651919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0786ff-b7f0-49f6-8823-a12830bc1e38 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:06.669273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:06.867157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:06.868096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:06.876254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:06.876360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users/1b0f367c561349f3a6e50b5ef79bd03d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:06.880807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1b0f367c561349f3a6e50b5ef79bd03d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:06.886822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:07.057212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.058121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.092754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.131522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.132769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0f367c561349f3a6e50b5ef79bd03d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.172688 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3555/7188] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:06 2022] DELETE /identity/v3/users/1b0f367c561349f3a6e50b5ef79bd03d => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:07.176663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:07.207473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad4015-8f3a-488f-b9f9-798eaeba3b48 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:07.319591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:07.321441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:07.328608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:07.416648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:07.417459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:07.417748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:07.418022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users/2676a4dfc54e48eab20ae56d7e4c8bf0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:07.419897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2676a4dfc54e48eab20ae56d7e4c8bf0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:07.422096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:07.475834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bbd98-8f82-420c-8e11-5f8cc3c766c3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:07.481913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:07.482766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:07.483114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:07.483461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] PATH_INFO: `/v3/users/99999871759d495a90a2ef21fc5c6969` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:07.484735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99999871759d495a90a2ef21fc5c6969)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:07.486924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:07.534300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:07.537120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.541008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.564455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.574148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99999871759d495a90a2ef21fc5c6969'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.575428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99999871759d495a90a2ef21fc5c6969'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.590236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.590537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99999871759d495a90a2ef21fc5c6969'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.592273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2676a4dfc54e48eab20ae56d7e4c8bf0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.619050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99999871759d495a90a2ef21fc5c6969'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.621814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6f9a79f5-60a9-48b6-8f87-e221d349f663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99999871759d495a90a2ef21fc5c6969'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:07.655998 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3634/7189] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:07 2022] DELETE /identity/v3/users/2676a4dfc54e48eab20ae56d7e4c8bf0 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:07.658992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:07.661941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:07.704301 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3556/7190] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:07 2022] DELETE /identity/v3/users/99999871759d495a90a2ef21fc5c6969 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:07.706584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:07.706865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:07.707137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:07.744190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:07.745518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:07.881546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fb92-54ff-41f8-b5cc-efafec9d7113 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:07.887701 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3557/7191] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:07.901895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:07.903734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:07.987282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:08.012865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.013540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.017101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.017101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/3986bda9c145457bb1b11dc192b8ca68` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.024284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3986bda9c145457bb1b11dc192b8ca68)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.029904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.096304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32def1-5f7d-4e5e-9372-6015274c839d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.105935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.106875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.107250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.107621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.159451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.161018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.189423 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3635/7192] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:07 2022] DELETE /identity/v3/groups/3986bda9c145457bb1b11dc192b8ca68 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:08.190651 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3558/7193] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:08.191374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.203734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:08.314237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb11c83-7ae0-4872-827d-b2f90d5a0e92 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.331108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.332067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.332549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.333289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.334119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.338014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.374633 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3636/7194] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:08 2022] POST /identity/v3/groups => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:08.375757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.394883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f134c-4238-4112-8237-a37fc1237cbf service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.403553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.404178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.404458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.404735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.405386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.407513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.436894 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3559/7195] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:08 2022] POST /identity/v3/groups => generated 327 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:08.441205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.457634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec88f1c-8c7c-45fc-818d-d9d4a978a362 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.470275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.470982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.471261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.471544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.476625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.478756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.518544 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3637/7196] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:08 2022] POST /identity/v3/groups => generated 326 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:08.523462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.536986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def2dc2a-b42a-406a-a031-a7881479c6a0 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.543264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.543887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.544168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.544438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.545097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.546726 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:08.546726 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:08.546726 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:08.548039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.569240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3560/7197] 10.222.0.52 () {58 vars in 1125 bytes} [Sat Sep 24 21:16:08 2022] GET /identity/v3/groups => generated 1490 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:08.574172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.597065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1da25-c057-4578-b091-9daf70aec419 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.615585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.623065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.623065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.623065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/fe038a61a8a84379b1db7771c58f6938` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.637434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=fe038a61a8a84379b1db7771c58f6938)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.648189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6349923b-2764-4512-a79e-bf3fbdc6ffd3 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:08.757037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3638/7198] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:08 2022] DELETE /identity/v3/groups/fe038a61a8a84379b1db7771c58f6938 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:08.758686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:08.766220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.766517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.766797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.783683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2edbc-24bb-43da-8ff7-6af126ff7b31 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:08.829934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] MFA Rules not processed for user `82c2160fcae947138ddd7b1b4df1e361`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:08.831226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:08.851771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:08.900820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:08.917452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:08.918463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:08.919116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:08.919549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/dc18099f9de5435a8634a42cac94b49d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:08.925631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3639/7199] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:08.929694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=dc18099f9de5435a8634a42cac94b49d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:08.931919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.064626 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3561/7200] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:08 2022] DELETE /identity/v3/groups/dc18099f9de5435a8634a42cac94b49d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:09.068936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.089499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae194c-b712-470c-893a-3e5e25aa66db None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:09.210866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:09.212031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.212502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.212929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/b4baf70c7c204f369099697c7aceedd1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.229630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=b4baf70c7c204f369099697c7aceedd1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:09.231644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.333289 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3640/7201] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:09 2022] DELETE /identity/v3/groups/b4baf70c7c204f369099697c7aceedd1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:09.339260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.385899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4a05-788a-4bbf-953f-219f8a4ac02b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:09.399770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.407229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:09.499476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:09.500745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.501285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.501759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.502523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:09.504665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980b35b-50a7-4a09-8803-b6e838122d58 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.549296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3562/7202] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:09 2022] POST /identity/v3/users => generated 323 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:09.549490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fae664-7599-4381-916f-73a78f7a18f6 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:09.549895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.549895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.549895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.558266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:09.558742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.559025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.559295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] PATH_INFO: `/v3/projects/c5b2658604ca463fb08b152900e01798` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.560472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b2658604ca463fb08b152900e01798)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:09.563102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.593234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] MFA Rules not processed for user `fc712fcf0f814a59bb52e16685ab1c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:09.595283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:09.660606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:09.666446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b2658604ca463fb08b152900e01798', 'request_id': 'req-f58156fb-70ee-4b62-a7d3-427b480ca68d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.674073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b2658604ca463fb08b152900e01798', 'request_id': 'req-f58156fb-70ee-4b62-a7d3-427b480ca68d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.685605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b2658604ca463fb08b152900e01798', 'request_id': 'req-f58156fb-70ee-4b62-a7d3-427b480ca68d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.695906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:09.705243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] The token cache is being invalidate because project c5b2658604ca463fb08b152900e01798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:09.705540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.706210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.711671 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3641/7203] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:09 2022] DELETE /identity/v3/projects/c5b2658604ca463fb08b152900e01798 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:09.711671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.711934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3563/7204] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:09.712031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:09.714034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.735954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc3e50-96e7-496a-9506-e9b4f1127813 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:09.815522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58156fb-70ee-4b62-a7d3-427b480ca68d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:09.822280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:09.823203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.823485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.823762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.824536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:09.829086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.857255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:09.858945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:09.859528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:09.860164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] PATH_INFO: `/v3/users/947800417f6d4a20ae2d27903dd6a5c7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:09.863142 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3642/7205] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:09 2022] POST /identity/v3/groups => generated 326 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:09.863503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:09.864237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947800417f6d4a20ae2d27903dd6a5c7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:09.868158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:09.868708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:09.991401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947800417f6d4a20ae2d27903dd6a5c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:09.992754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947800417f6d4a20ae2d27903dd6a5c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:10.012901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947800417f6d4a20ae2d27903dd6a5c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:10.028448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947800417f6d4a20ae2d27903dd6a5c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:10.030010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947800417f6d4a20ae2d27903dd6a5c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:10.055430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44368fe-c939-4256-b1a4-03a9eac8a46a tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:10.081713 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3564/7206] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:09 2022] DELETE /identity/v3/users/947800417f6d4a20ae2d27903dd6a5c7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:10.082713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:10.089235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:10.098883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:10.104631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.105428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.106999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.111239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:10.258908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd998-1682-4e66-9de5-9c363167cfb2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:10.287236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:10.288506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.288795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.289073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/d571538ad7aa4677ad699fcd275d863b/users/1380e78c73694991a769fea11c940f1d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.300167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:10.301935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea8414-580c-4a5b-9cbe-fe03830ff802 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:10.314651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:10.332140 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3643/7207] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:10.364944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=d571538ad7aa4677ad699fcd275d863b, user_id=1380e78c73694991a769fea11c940f1d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:10.366983 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:10.366983 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:10.366983 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:10.368228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:10.378225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:10.433210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.433955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.434425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.450774 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3565/7208] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:10 2022] PUT /identity/v3/groups/d571538ad7aa4677ad699fcd275d863b/users/1380e78c73694991a769fea11c940f1d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:10.454756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:10.503903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] MFA Rules not processed for user `992bb56657f340919192b7624025e253`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:10.505231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:10.593331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51a0ed-5254-4b3b-a776-06d38f4c3628 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:10.601850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:10.602827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.603244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.603725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.604473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:10.608701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:10.642781 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3566/7209] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:10 2022] POST /identity/v3/groups => generated 326 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:10.650101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:10.667822 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3644/7210] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:10.670253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984f175-b2ee-41f2-8f05-553a3f25c739 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:10.675190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:10.675918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.676206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.676481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/c7140af45483491b815274752ea969a6/users/1380e78c73694991a769fea11c940f1d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.684235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=c7140af45483491b815274752ea969a6, user_id=1380e78c73694991a769fea11c940f1d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:10.686260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:10.723071 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3567/7211] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:10 2022] PUT /identity/v3/groups/c7140af45483491b815274752ea969a6/users/1380e78c73694991a769fea11c940f1d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:10.727855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:10.740948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093ede1-d9db-4cbc-96d7-48b6f0c27900 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:10.898444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:10.899287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:10.900961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:10.901211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users/1380e78c73694991a769fea11c940f1d/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:10.904359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=1380e78c73694991a769fea11c940f1d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:10.936446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:10.949299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:10.956345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:10.991049 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3645/7212] 10.222.0.52 () {58 vars in 1242 bytes} [Sat Sep 24 21:16:10 2022] GET /identity/v3/users/1380e78c73694991a769fea11c940f1d/groups => generated 838 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:10.994934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:11.036000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999130a-0554-4965-98c0-fa8df0d22633 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.048303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.049320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.049609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.049884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/c7140af45483491b815274752ea969a6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.054059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=c7140af45483491b815274752ea969a6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.060286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.119597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d52aff-ff05-4134-b842-a360150661c7 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.127693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.132119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.132574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.132739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] PATH_INFO: `/v3/projects/392f324f6e7f44bbab592758663edb82` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.135155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392f324f6e7f44bbab592758663edb82)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.137090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.188860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3646/7213] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:10 2022] DELETE /identity/v3/groups/c7140af45483491b815274752ea969a6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:11.190812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:11.270044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392f324f6e7f44bbab592758663edb82', 'request_id': 'req-30dd2e40-42a7-4243-9d25-59d833dace04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:11.291387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:11.297094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392f324f6e7f44bbab592758663edb82', 'request_id': 'req-30dd2e40-42a7-4243-9d25-59d833dace04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:11.313501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392f324f6e7f44bbab592758663edb82', 'request_id': 'req-30dd2e40-42a7-4243-9d25-59d833dace04'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:11.328503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:11.344745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] The token cache is being invalidate because project 392f324f6e7f44bbab592758663edb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:11.345051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:11.346926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-30dd2e40-42a7-4243-9d25-59d833dace04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:11.355944 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3568/7214] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:10 2022] DELETE /identity/v3/projects/392f324f6e7f44bbab592758663edb82 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:11.358593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402be58-78e8-4f84-a993-729710948fc2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.358893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402be58-78e8-4f84-a993-729710948fc2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.359183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402be58-78e8-4f84-a993-729710948fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.365452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8f69-e230-4194-aac2-9cbea1227359 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.400182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9402be58-78e8-4f84-a993-729710948fc2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:11.402129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9402be58-78e8-4f84-a993-729710948fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:11.544439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.545031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.545314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.545586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/groups/d571538ad7aa4677ad699fcd275d863b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.547351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_group(group_id=d571538ad7aa4677ad699fcd275d863b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.557224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.557686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3569/7215] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:11.560093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.560377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.560653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.627022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:11.633051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:11.657739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3570/7216] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:11.662325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:11.680113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b3bae-161a-4c61-84fc-3a0aac7f3b33 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.680973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3647/7217] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:11 2022] DELETE /identity/v3/groups/d571538ad7aa4677ad699fcd275d863b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:11.685330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:11.689927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:11.830196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.830925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.831486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.831758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.832467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:11.832763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.834831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.852688 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3571/7218] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:11.857345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:11.872486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17b502-f70e-42c4-aff8-b1b2d9f5644a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.880551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec259d35-cad5-4ba9-ba61-b8a85b133911 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:11.885926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.887549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.887844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.888170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.888971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.894413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.900900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:11.903271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:11.904188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:11.904638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/users/1380e78c73694991a769fea11c940f1d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:11.908134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1380e78c73694991a769fea11c940f1d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:11.912155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:11.983092 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3572/7219] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:11 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:11.987790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.039176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1380e78c73694991a769fea11c940f1d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.040755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1380e78c73694991a769fea11c940f1d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.045783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:12.055628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1380e78c73694991a769fea11c940f1d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.086989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1380e78c73694991a769fea11c940f1d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.089449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1380e78c73694991a769fea11c940f1d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.099974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:12.122167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3648/7220] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:11 2022] DELETE /identity/v3/users/1380e78c73694991a769fea11c940f1d => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:12.125564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c36e5-5f7c-4584-a8b1-f95d1600b0d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.126853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.128396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:12.262558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.262558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.262558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.262558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.263108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.263108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.300758 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3573/7221] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:11 2022] POST /identity/v3/users => generated 289 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:12.301883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.305341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:12.319385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6ad0-a69a-4f17-8e6e-f63c642cbd1f tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.321585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdda26-89bf-4284-9b8d-fef987a79f2c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.327606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.328296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.328584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.328856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.329500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.331898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.340735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.341745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.342891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.343514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/domains/2ab951e469b54b168ceef387a0b9c73d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.345055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=2ab951e469b54b168ceef387a0b9c73d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.347075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.349440 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3574/7222] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:12.355052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.368336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96386618-b8a3-4899-9102-1420a99251cd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.374006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.375054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.375335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.375607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.376387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.378156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.392666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.393537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.408648 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3575/7223] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:12.412270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.417809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:12.449125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.450820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.455951 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3649/7224] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:16:12 2022] PATCH /identity/v3/domains/2ab951e469b54b168ceef387a0b9c73d => generated 277 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:12.462382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4cbc-7e54-4f46-868e-5b8f765a02a8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.469449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:12.476052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:12.588895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e556e3d-d3fe-4450-a900-2ceec2890071 tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:12.604088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d0eab16387740d1b6d7ee844bef1cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56f21092159b4fe4a8753788bbe9f3e2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.604824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.605644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.606941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] PATH_INFO: `/v3/domains/2ab951e469b54b168ceef387a0b9c73d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.630689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:12.632014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.632367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.632979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] PATH_INFO: `/v3/projects/3601e0eb4f654079810627bef64cb0da/users/b53b6cad57584720aa0ecbe097fe92c9/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.633521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=2ab951e469b54b168ceef387a0b9c73d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.639267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.664357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3601e0eb4f654079810627bef64cb0da, user_id=b53b6cad57584720aa0ecbe097fe92c9, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:12.667281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e06b-ab4c-4f2a-a741-5800759fd4a8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:12.702975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3576/7225] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:12 2022] PUT /identity/v3/projects/3601e0eb4f654079810627bef64cb0da/users/b53b6cad57584720aa0ecbe097fe92c9/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:12.710531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:12.710728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:12.710983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:12.712353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.745512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.748919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:12.775613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:12.777127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:12.884841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.901246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.927179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:12.943471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:12.999912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] The token cache is being invalidate because project 2ab951e469b54b168ceef387a0b9c73d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:13.000644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.004057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.021683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.027966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.028777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '2ab951e469b54b168ceef387a0b9c73d', 'request_id': 'req-689bc078-74a5-4291-9f9b-199cc343236b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.033132 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3577/7226] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:13.042444 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3650/7227] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:12 2022] DELETE /identity/v3/domains/2ab951e469b54b168ceef387a0b9c73d => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:13.043918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:13.050019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:13.050637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:13.054987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:13.080989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c606a-5146-46e2-aa71-76926424b9eb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:13.255739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bc078-74a5-4291-9f9b-199cc343236b tempest-GroupsV3TestJSON-2138384090 tempest-GroupsV3TestJSON-2138384090-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:13.261812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:13.262385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:13.262844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:13.263123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:13.265951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:13.281202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:13.281849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:13.282135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:13.282406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] PATH_INFO: `/v3/users/39a60695d08a4ae79d776a4ccdf4b348` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:13.287116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a60695d08a4ae79d776a4ccdf4b348)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:13.291512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:13.403613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a60695d08a4ae79d776a4ccdf4b348'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.404450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a60695d08a4ae79d776a4ccdf4b348'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.418862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:13.421098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a60695d08a4ae79d776a4ccdf4b348'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.423987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e303ed-62c5-4c77-a127-2872071e3caf service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:13.436346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a60695d08a4ae79d776a4ccdf4b348'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.437896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a60695d08a4ae79d776a4ccdf4b348'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:13.454629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3651/7228] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:16:13.494239 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3578/7229] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:13 2022] DELETE /identity/v3/users/39a60695d08a4ae79d776a4ccdf4b348 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:13.498708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:13.499017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:13.499319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:13.536273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:13.537766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:13.594063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:13.745329 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3652/7230] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:13.790292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:13.791986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:13.916160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918e88d-0bcf-40d4-9dcb-90bafd6580a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:13.925152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:13.925152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:13.925152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:13.925152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:13.948892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:13.950918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:13.979912 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3579/7231] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:14.719223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:14.720766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:14.743205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad316a5-2f2e-4d82-b74c-91b498c94e6a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:14.756989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:14.757777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:14.758066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:14.758445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] PATH_INFO: `/v3/projects/c22cf4343bb04e44afe9664484ecbf02` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:14.760040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22cf4343bb04e44afe9664484ecbf02)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:14.762759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:14.776688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:14.791759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b7d4d-0bce-4bd5-9d7e-728aeb75006a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:14.802598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:14.805327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:14.806628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:14.809155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:14.814194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:14.853753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22cf4343bb04e44afe9664484ecbf02', 'request_id': 'req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:14.855162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:14.865039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22cf4343bb04e44afe9664484ecbf02', 'request_id': 'req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:14.872270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:14.875892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d303fc-8d9b-48bd-bea3-83bac4a70d38 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:14.878012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22cf4343bb04e44afe9664484ecbf02', 'request_id': 'req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:14.898471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] The token cache is being invalidate because project c22cf4343bb04e44afe9664484ecbf02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:14.899036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:14.901174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:14.903471 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3653/7232] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:14 2022] DELETE /identity/v3/projects/c22cf4343bb04e44afe9664484ecbf02 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:14.907232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:14.909504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:14.920093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3580/7233] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:15.102475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7fd42-d59d-4af3-9b01-1ddce46a150e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:15.118004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:15.118967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:15.119358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:15.119734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] PATH_INFO: `/v3/users/9d0eab16387740d1b6d7ee844bef1cdd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:15.122799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0eab16387740d1b6d7ee844bef1cdd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:15.127245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:15.272641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:15.272924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:15.273286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:15.285364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0eab16387740d1b6d7ee844bef1cdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:15.288444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0eab16387740d1b6d7ee844bef1cdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:15.296144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:15.314410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0eab16387740d1b6d7ee844bef1cdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:15.343850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] MFA Rules not processed for user `b53b6cad57584720aa0ecbe097fe92c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:15.345171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:15.367051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0eab16387740d1b6d7ee844bef1cdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:15.368281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0eab16387740d1b6d7ee844bef1cdd'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:15.410105 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3654/7234] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:14 2022] DELETE /identity/v3/users/9d0eab16387740d1b6d7ee844bef1cdd => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:15.497731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3581/7235] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:15.502622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:15.504209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:15.673477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311d892-8df2-4eda-a855-e66abae579cd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:15.689488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:15.690132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:15.690421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:15.690845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:15.691507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:15.693590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:15.758387 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3655/7236] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:15 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:15.763248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:15.780985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209968ed-3c99-4ee1-a825-a2c38ef19c35 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:15.966780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:15.976109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:15.978005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:15.978128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:15.980879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:15.984074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:15.985058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:15.993273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.056740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3582/7237] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:15 2022] POST /identity/v3/users => generated 296 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:16.057715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.102338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3957-ad4d-4839-9b55-7e9beb07c36f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.140871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.141927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.143059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.144763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.146410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.152601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.164789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b21345-0a73-44be-b9e6-5bf749c33fc4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.174100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.174711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.174992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.175262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.177039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:16.202876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.204770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.213673 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3583/7238] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:16.218641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.236566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19dc50-1a3b-4b91-8f19-7fb3c9d8f77c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.237578 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3656/7239] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:16.241439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.243244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:16.249642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.250654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.251583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.251583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] PATH_INFO: `/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f/users/d872d0d2c09344e7b9edbf8b2065e0a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.275954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d4ad2-20c0-4736-b33d-26d1189b2867 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.280248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1bf27d27e2490592d10c5f8c15d75f, user_id=d872d0d2c09344e7b9edbf8b2065e0a4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.285302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.285950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.286235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.286522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] PATH_INFO: `/v3/projects/56f21092159b4fe4a8753788bbe9f3e2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.288146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f21092159b4fe4a8753788bbe9f3e2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.290060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.290781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.326113 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3584/7240] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:16 2022] PUT /identity/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f/users/d872d0d2c09344e7b9edbf8b2065e0a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:16.327248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.380816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:INSERT=2,keystone:DELETE=5,keystone:SELECT=93 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:16.420079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f21092159b4fe4a8753788bbe9f3e2', 'request_id': 'req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:16.428701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:16.457913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f21092159b4fe4a8753788bbe9f3e2', 'request_id': 'req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:16.470972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f21092159b4fe4a8753788bbe9f3e2', 'request_id': 'req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:16.500587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] The token cache is being invalidate because project 56f21092159b4fe4a8753788bbe9f3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:16.500916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:16.513231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234be9-319c-406e-a76f-d348c8749f02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.514918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1adbfa94-6efe-4df6-aa6c-cb2e374f9fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:16.533432 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3657/7241] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:16 2022] DELETE /identity/v3/projects/56f21092159b4fe4a8753788bbe9f3e2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:16.538193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4830f-6d60-42d8-ae32-4cfbd42b843b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.538566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4830f-6d60-42d8-ae32-4cfbd42b843b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.542463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4830f-6d60-42d8-ae32-4cfbd42b843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.598138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-99a4830f-6d60-42d8-ae32-4cfbd42b843b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:16.601059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99a4830f-6d60-42d8-ae32-4cfbd42b843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:16.724742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.725984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.726261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.726678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.728996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.732645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.754559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3585/7242] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:16.757377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.760338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:16.783169 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3658/7243] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:16.784737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcffdf-536e-48ba-a524-a8baca79bfc4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.792008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.792393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.792772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.796039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.798105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.798479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.798779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] PATH_INFO: `/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f/users/d872d0d2c09344e7b9edbf8b2065e0a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.801344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1bf27d27e2490592d10c5f8c15d75f, user_id=d872d0d2c09344e7b9edbf8b2065e0a4, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.804616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.825180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3586/7244] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:16 2022] PUT /identity/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f/users/d872d0d2c09344e7b9edbf8b2065e0a4/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:16.835799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:16.837731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:16.918949 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3659/7245] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:16.923485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.944676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d226fc-335c-45a4-9be8-9c8aa4510064 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:16.950943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:16.951703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:16.952326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:16.953490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:16.954230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:16.954659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:16.956452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:16.973749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3587/7246] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:16.976752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:16.987611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17975bbf-ad40-41f7-b699-3c108051a507 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.005157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.007827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.008697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.009803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.010687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.013288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.051451 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3660/7247] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:16 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:17.053051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.074555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9997d46-366a-4bb9-83cb-0ff8f1020929 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.172709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.173496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.173786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.174062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.174951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.177402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.218990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.219311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3588/7248] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:17 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:17.238636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4be46-80f1-448c-a945-cb441c75d237 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.254621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.255974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.256830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.257110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.260347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.262044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.286555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3661/7249] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:17.289210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.305756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a02a33-a883-4432-9363-01547a13ef13 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.312518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.313348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.313732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.315672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.316193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.319579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78b0a3-b127-4427-93ab-481d7a774ea9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.335867 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3589/7250] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:17.340121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.384424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb37-dc9e-42a4-9585-242f2d0719da admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.405738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.406649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.406943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.407235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] PATH_INFO: `/v3/projects/dcee9098e251401cb75596f6d35557b5/users/0dcae982b25043ffa7b8073a9bba3eab/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.434880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcee9098e251401cb75596f6d35557b5, user_id=0dcae982b25043ffa7b8073a9bba3eab, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.438477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.456795 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3662/7251] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:17 2022] PUT /identity/v3/projects/dcee9098e251401cb75596f6d35557b5/users/0dcae982b25043ffa7b8073a9bba3eab/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:17.471656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.471959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.472249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.521706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] MFA Rules not processed for user `0dcae982b25043ffa7b8073a9bba3eab`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:17.523174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:17.555909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:17.609138 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3590/7252] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:17.614696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.620047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:17.708008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353cf88-e9e0-4900-99f7-f807422ed8ac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.713794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.714401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.714833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.715105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.715747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.717708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.749140 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3663/7253] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:17 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:17.751573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.764867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b56197-bbc1-471b-ab0b-b276fab0f35e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.830878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.831598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.831770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.832018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.832726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.835191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.860850 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3591/7254] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:17 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:17.864394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.876152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a77fb-98b6-45c3-ac8c-aec7566af5db admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.883667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.884781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.885423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.885920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.886889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.890318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.905596 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3664/7255] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:17.909739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.923661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66271a06-9920-42c5-8040-98d6a6b9813d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:17.931379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:17.932820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:17.933108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:17.933383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] PATH_INFO: `/v3/projects/da64abe970de4712a0af625511bd15a3/users/0bff275901ae414f9b483b5bf7dd2589/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:17.955528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da64abe970de4712a0af625511bd15a3, user_id=0bff275901ae414f9b483b5bf7dd2589, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:17.958567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:17.976032 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3592/7256] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:17 2022] PUT /identity/v3/projects/da64abe970de4712a0af625511bd15a3/users/0bff275901ae414f9b483b5bf7dd2589/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:17.980154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:17.990711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db2578-6a2c-46a2-bda9-52ac2346499a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.060903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.061535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.061854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.062691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.062900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.065111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32317a-d0d5-46c1-8971-0bd210c6b039 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.081718 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3665/7257] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:18.083402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.103021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f2bb0-22c2-4ff0-897a-6dc7353404d0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.117863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.118582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.118863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.119138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] PATH_INFO: `/v3/projects/da64abe970de4712a0af625511bd15a3/users/0bff275901ae414f9b483b5bf7dd2589/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.121107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da64abe970de4712a0af625511bd15a3, user_id=0bff275901ae414f9b483b5bf7dd2589, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.123995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.146679 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3593/7258] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:18 2022] PUT /identity/v3/projects/da64abe970de4712a0af625511bd15a3/users/0bff275901ae414f9b483b5bf7dd2589/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:18.157348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.157727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.158551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.204746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] MFA Rules not processed for user `0bff275901ae414f9b483b5bf7dd2589`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:18.206967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:18.332846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:18.341585 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3666/7259] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:18.346038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.357537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eed1-bdc2-4784-884e-7720320ec071 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.363532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.364267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.364539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.364889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects/dcee9098e251401cb75596f6d35557b5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.366835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:get_project(project_id=dcee9098e251401cb75596f6d35557b5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.369123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.371877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3594/7260] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects/dcee9098e251401cb75596f6d35557b5 => generated 387 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:18.376434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.388595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4095463-c3a7-4871-984f-cc5cf3ff0b80 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.399349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.400100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.400446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.400819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.401820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.403949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053e5ec-afd6-4245-bbb4-55c5d4837933 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.455903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3667/7261] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:16:18 2022] POST /identity/v3/projects => generated 339 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:18.456648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.469930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4923af-c217-429b-8c3e-fcf05c25a121 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.470283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4923af-c217-429b-8c3e-fcf05c25a121 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.470577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4923af-c217-429b-8c3e-fcf05c25a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.478756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4630ea-6c85-45ea-ad83-0fef86b7276a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.505687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc4923af-c217-429b-8c3e-fcf05c25a121 None None] MFA Rules not processed for user `d872d0d2c09344e7b9edbf8b2065e0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:18.507542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc4923af-c217-429b-8c3e-fcf05c25a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:18.604433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.605081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.605950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.605950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.606757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.609031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.636664 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3595/7262] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects => generated 7346 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:18.641627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.677618 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3668/7263] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:18.677724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36447e-1b7c-49c2-b08b-7f1b124c122a tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.681032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.681438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.681808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.696698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.697394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.697674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.697945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects/dcee9098e251401cb75596f6d35557b5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.699370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:get_project(project_id=dcee9098e251401cb75596f6d35557b5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.701343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.705573 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3596/7264] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects/dcee9098e251401cb75596f6d35557b5 => generated 387 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:18.707280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.712430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] MFA Rules not processed for user `b53b6cad57584720aa0ecbe097fe92c9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:18.713992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:18.729630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:18.744147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693535ee-e13b-4acd-971f-5db36994ddee tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.759442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.760120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.760469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.760745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects/0b2a6044fb974bc394379acb86e76fbd` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.761900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:get_project(project_id=0b2a6044fb974bc394379acb86e76fbd)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.764384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.773231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3597/7265] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects/0b2a6044fb974bc394379acb86e76fbd => generated 339 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:18.774187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.793138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5e9a9-aa30-4e3b-ba04-488cec4326d4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.798801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.799408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.799688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.799960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.800595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.802780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.815773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3669/7266] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:18.828997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.834764 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3598/7267] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects => generated 7346 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:18.836094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:18.843879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.857653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec160ac7-d448-4980-a9f6-b2a0734c88af tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.862932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.863706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.864138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.864598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.865472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:18.866594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.868805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.900886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3599/7268] 10.222.0.52 () {58 vars in 1166 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects?domain_id=default => generated 7015 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:18.902254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.913272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afae90-151a-4d98-9a4b-1e5cab217b85 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.920355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.921085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.921606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.921993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.922792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.929355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:18.931014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.936110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed297e03-43d8-46a9-8e6a-d16fdd97d67f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.944608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.945333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.945727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.946010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.948427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:18.954645 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3600/7269] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects => generated 7346 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:18.958406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:18.963119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.965602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:18.974429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc6d83-62cd-49ce-b63e-f38f89b55fb4 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:18.982314 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3670/7270] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:18.983029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:18.983794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:18.984081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:18.986996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:18.987702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:18.988059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:18.990014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.013927 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3601/7271] 10.222.0.52 () {58 vars in 1166 bytes} [Sat Sep 24 21:16:18 2022] GET /identity/v3/projects?domain_id=default => generated 7015 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:19.021180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:19.033862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbdd7a-d6a4-4286-bcd6-9f17bbb07508 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:19.040738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:19.041764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.041764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.041967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.042588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:19.044584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.065128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3671/7272] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:16:19 2022] GET /identity/v3/projects => generated 7346 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:19.071306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:19.088220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e3001-972c-4112-b50c-74a66fb4b037 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:19.094285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:19.095050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.095340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.095616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.096288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-578043450) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:19.096592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:19.098538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.125966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:19.126422 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3602/7273] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:19 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-578043450 => generated 534 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:19.139256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298c765-4ac1-48cf-b849-9218abf5f54c tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:19.144428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bff275901ae414f9b483b5bf7dd2589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da64abe970de4712a0af625511bd15a3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:19.145080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.145358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.145630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] PATH_INFO: `/v3/projects/0b2a6044fb974bc394379acb86e76fbd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.148738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0b2a6044fb974bc394379acb86e76fbd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:19.152004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.208013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2a6044fb974bc394379acb86e76fbd', 'request_id': 'req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.214706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2a6044fb974bc394379acb86e76fbd', 'request_id': 'req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.221016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2a6044fb974bc394379acb86e76fbd', 'request_id': 'req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.234668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] The token cache is being invalidate because project 0b2a6044fb974bc394379acb86e76fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:19.234958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.235606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cf0be0ef-a489-4cde-aa19-1f14d7c7d959 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.238264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3672/7274] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:19 2022] DELETE /identity/v3/projects/0b2a6044fb974bc394379acb86e76fbd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:19.242794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:19.244643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:19.324882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a767b8-7dc9-4fd0-8764-23d7372dcdb1 tempest-ListProjectsStaticTestJSON-1597759525 tempest-ListProjectsStaticTestJSON-1597759525-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:19.391496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:19.392140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.392420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.392697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] PATH_INFO: `/v3/users/0dcae982b25043ffa7b8073a9bba3eab` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.393838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcae982b25043ffa7b8073a9bba3eab)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:19.395810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.445029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:19.452872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcae982b25043ffa7b8073a9bba3eab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.453771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcae982b25043ffa7b8073a9bba3eab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.461587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcae982b25043ffa7b8073a9bba3eab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.469114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcae982b25043ffa7b8073a9bba3eab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.469816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcae982b25043ffa7b8073a9bba3eab'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:19.488269 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3603/7275] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:19 2022] DELETE /identity/v3/users/0dcae982b25043ffa7b8073a9bba3eab => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:19.493268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.493626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.493979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.515244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:19.516574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:19.582150 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3673/7276] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:19.591812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:19.593189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:19.671553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc04fd-b1f2-4f30-8ae1-7036780b950e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:19.675648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:19.676258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:19.676543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:19.676816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:19.689355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:19.690573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:19.710634 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3604/7277] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:20.308953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:20.310844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:20.340281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c1c47-5a0f-4a3e-8730-8521938a16ca None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:20.359941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:20.360595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:20.361314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:20.361314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] PATH_INFO: `/v3/projects/dcee9098e251401cb75596f6d35557b5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:20.362417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcee9098e251401cb75596f6d35557b5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:20.364943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:20.440858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcee9098e251401cb75596f6d35557b5', 'request_id': 'req-6639f4aa-f6c9-4661-b621-49302b5c6b6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.458146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:20.460818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcee9098e251401cb75596f6d35557b5', 'request_id': 'req-6639f4aa-f6c9-4661-b621-49302b5c6b6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.476989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcee9098e251401cb75596f6d35557b5', 'request_id': 'req-6639f4aa-f6c9-4661-b621-49302b5c6b6d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.499679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] The token cache is being invalidate because project dcee9098e251401cb75596f6d35557b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:20.500050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.500888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.507407 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3674/7278] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:20 2022] DELETE /identity/v3/projects/dcee9098e251401cb75596f6d35557b5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:20.508479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:20.520734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6532f9-c36f-4233-baf9-c6882149ba49 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:20.617574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:20.618216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:20.618659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:20.618970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] PATH_INFO: `/v3/users/0bff275901ae414f9b483b5bf7dd2589` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:20.620249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bff275901ae414f9b483b5bf7dd2589)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:20.622180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:20.686291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bff275901ae414f9b483b5bf7dd2589'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.686996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bff275901ae414f9b483b5bf7dd2589'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.697189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bff275901ae414f9b483b5bf7dd2589'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.707008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bff275901ae414f9b483b5bf7dd2589'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.708067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-678acad2-76ba-431b-abed-fa40d666c38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bff275901ae414f9b483b5bf7dd2589'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:20.710189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:20.735403 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3605/7279] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:20 2022] DELETE /identity/v3/users/0bff275901ae414f9b483b5bf7dd2589 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:21.065389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.067104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:21.116907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639f4aa-f6c9-4661-b621-49302b5c6b6d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.255873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.258071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.258539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.259087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] PATH_INFO: `/v3/projects/da64abe970de4712a0af625511bd15a3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.260629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=da64abe970de4712a0af625511bd15a3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.264031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.352784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da64abe970de4712a0af625511bd15a3', 'request_id': 'req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:21.358585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da64abe970de4712a0af625511bd15a3', 'request_id': 'req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:21.364853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da64abe970de4712a0af625511bd15a3', 'request_id': 'req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:21.380761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] The token cache is being invalidate because project da64abe970de4712a0af625511bd15a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:21.382064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:21.382381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-bcde3e4d-81b1-4f3d-adce-a4f56aa4cb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:21.384166 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3675/7280] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:21 2022] DELETE /identity/v3/projects/da64abe970de4712a0af625511bd15a3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:21.389071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.389355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.389635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.410726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:21.411944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:21.476149 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3606/7281] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:21.483874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080d34-72da-42df-89a6-2387e468cace None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.484162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080d34-72da-42df-89a6-2387e468cace None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.484437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080d34-72da-42df-89a6-2387e468cace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.504377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3080d34-72da-42df-89a6-2387e468cace None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:21.505601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3080d34-72da-42df-89a6-2387e468cace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:21.514273 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3676/7282] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:21.518902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.530118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cc0f1-4f65-40a7-b1ed-6e64e6ee1b6a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.535442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.536116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.536464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.536806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.537789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:21.538166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.539834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.551555 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3607/7283] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:21.556775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3080d34-72da-42df-89a6-2387e468cace None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.581650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3080d34-72da-42df-89a6-2387e468cace None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.588133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.588764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.589892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.589892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.590184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.592448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.635163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3677/7284] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:21 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:21.639362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.653437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690db000-c430-49f8-90ca-dbd53e7ca67e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.733845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.734462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.734781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.735059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.735699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.737839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.770003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3608/7285] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:21 2022] POST /identity/v3/users => generated 283 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:21.770847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.790939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd077987-f70c-46dd-a28a-454b9d30f969 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.797659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.798351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.798993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.799276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.799959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.801592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.814237 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3678/7286] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:21.818115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.829105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bdb23-20a2-4397-950e-977f939cc8a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.834306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.835016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.835466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.835748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.836476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.838109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0585eeb-0565-48bf-a976-a1d92f87710a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.853961 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3609/7287] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:21.855230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:21.868304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029adec7-bf4b-4dc5-adc0-d12c5209039a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:21.874116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:21.874960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.875305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.875652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] PATH_INFO: `/v3/projects/f147d87eb6624ba99b1875c87d127011/users/4d0506932e8a4df5a5e30539840cd300/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.893896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f147d87eb6624ba99b1875c87d127011, user_id=4d0506932e8a4df5a5e30539840cd300, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:21.897561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:21.914564 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3679/7288] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:21 2022] PUT /identity/v3/projects/f147d87eb6624ba99b1875c87d127011/users/4d0506932e8a4df5a5e30539840cd300/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:21.926383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:21.926712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:21.926990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:21.984695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] MFA Rules not processed for user `4d0506932e8a4df5a5e30539840cd300`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:21.988112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:22.059579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:22.116799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.117808 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3610/7289] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:22.137569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:22.214685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57e80-9bef-4e3c-877e-cd80c33f9bc8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.222479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.223265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.223628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.224286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.225709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.228346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.270572 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3680/7290] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:22 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:22.274256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.286342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6d4b4-5d70-40b0-894a-09e9f9122af9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.395598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.396223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.396620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.397476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.398923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.401723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.433695 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3611/7291] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:22 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:22.438018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.457010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806205c-4e20-468a-9679-e34f27b63696 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.464079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.464725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.465002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.465271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.465908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.483632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.503264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3681/7292] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:22.504147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.517783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910b8da-f359-4a7b-937d-2adec0860358 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.526560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.527887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.528417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.529233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] PATH_INFO: `/v3/projects/616dafbd4a014451ab951bf0eba31d7f/users/06228f6dd02641ebb8931b76e8e248b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.552947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616dafbd4a014451ab951bf0eba31d7f, user_id=06228f6dd02641ebb8931b76e8e248b0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.557595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.590674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3612/7293] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:22 2022] PUT /identity/v3/projects/616dafbd4a014451ab951bf0eba31d7f/users/06228f6dd02641ebb8931b76e8e248b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:22.594170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.612353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0538b1-88ec-4986-9c45-6787267a6bdc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.691919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.692473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.692755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.693404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.695102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.697757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53e981-d536-48ac-afb4-481f599d3314 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.711571 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3682/7294] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:22.712791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.724482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a790887-cbf9-4678-8520-1fd99385aa47 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.731648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.732756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.732756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.732938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] PATH_INFO: `/v3/projects/616dafbd4a014451ab951bf0eba31d7f/users/06228f6dd02641ebb8931b76e8e248b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.735845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616dafbd4a014451ab951bf0eba31d7f, user_id=06228f6dd02641ebb8931b76e8e248b0, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.738730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.752654 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3613/7295] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:22 2022] PUT /identity/v3/projects/616dafbd4a014451ab951bf0eba31d7f/users/06228f6dd02641ebb8931b76e8e248b0/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:22.764847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.765601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.766080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.807827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] MFA Rules not processed for user `06228f6dd02641ebb8931b76e8e248b0`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:22.809680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:22.885009 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3683/7296] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:22.889494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.900721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad211c8-a077-40d1-85b0-017c8b8bbaa6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.906261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:22.906407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:22.906775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:22.907054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:22.907741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:22.909338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:22.928205 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3614/7297] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:16:22 2022] POST /identity/v3/domains => generated 275 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:22.932360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:22.942020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c9173-5339-4d3e-bbe4-b9df189c8c26 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:22.948335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:23.007803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.008421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.008708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.008993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.009849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.011829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.049502 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3684/7298] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:22 2022] POST /identity/v3/users => generated 348 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:23.050729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.062525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5789ee-6add-4cfa-b08b-b15e96d05bd5 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.067362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.067970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.068283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.068588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.069205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.071108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.100740 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3615/7299] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:23 2022] POST /identity/v3/users => generated 323 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:23.101924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.112318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50583-75f3-4a05-8806-abde14fb069b tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.117959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.118623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.118908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.119186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users/230b4dcac432471e99d84c05bb709880` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.135244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:get_user(user_id=230b4dcac432471e99d84c05bb709880)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.137418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.143321 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3685/7300] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:16:23 2022] GET /identity/v3/users/230b4dcac432471e99d84c05bb709880 => generated 348 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:23.145882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.159778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dffee9-4656-4cb7-af48-b9878eced482 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.168061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.168061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.168061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.168472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.169092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Adding query filter params (domain_id=141a98759c7e470e8c4d0f19ecefb1cb) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:23.169623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.172270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.185723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3616/7301] 10.222.0.52 () {58 vars in 1207 bytes} [Sat Sep 24 21:16:23 2022] GET /identity/v3/users?domain_id=141a98759c7e470e8c4d0f19ecefb1cb => generated 485 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:23.190593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.201176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147f429-7873-4da5-b63d-92331ff448fb tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.208087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.208773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.209470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.209711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.210589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.212629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.256629 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3686/7302] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:23 2022] GET /identity/v3/users => generated 9187 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:23.259160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.272623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829192-cf7f-478f-9c93-9ee3110d1dba tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.277658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.278231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.278546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.278873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.279582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Adding query filter params (name=tempest-test_user-1922565092) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:23.279884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.282168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.329960 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3617/7303] 10.222.0.52 () {58 vars in 1189 bytes} [Sat Sep 24 21:16:23 2022] GET /identity/v3/users?name=tempest-test_user-1922565092 => generated 476 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:23.335190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.356478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669d826-4028-428b-b60e-7ee2ba5b72d1 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.361369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.361976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.362259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.362567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.363269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:23.363577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.366985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.393138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3687/7304] 10.222.0.52 () {58 vars in 1149 bytes} [Sat Sep 24 21:16:23 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:23.398525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.411751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d368c-664c-467e-9cc6-36301851f510 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.423162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.423833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.424135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.424391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users/04caa24a78c249a389e5623d1bc241b5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.445230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:delete_user(user_id=04caa24a78c249a389e5623d1bc241b5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.447941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.475808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=9,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:23.533426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04caa24a78c249a389e5623d1bc241b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.535154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04caa24a78c249a389e5623d1bc241b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.568747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04caa24a78c249a389e5623d1bc241b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.591418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04caa24a78c249a389e5623d1bc241b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.592396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04caa24a78c249a389e5623d1bc241b5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.668136 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3618/7305] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:23 2022] DELETE /identity/v3/users/04caa24a78c249a389e5623d1bc241b5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:23.671984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.673933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:23.701739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291d486-f08e-43bc-8847-61f6a44bfa48 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:23.801587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:23.802356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:23.802826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:23.803897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/users/230b4dcac432471e99d84c05bb709880` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:23.805417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:delete_user(user_id=230b4dcac432471e99d84c05bb709880)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:23.807791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:23.860008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230b4dcac432471e99d84c05bb709880'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.860988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230b4dcac432471e99d84c05bb709880'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.866775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:23.875702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230b4dcac432471e99d84c05bb709880'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.893277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230b4dcac432471e99d84c05bb709880'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.894042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230b4dcac432471e99d84c05bb709880'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:23.907339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3688/7306] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:23 2022] DELETE /identity/v3/users/230b4dcac432471e99d84c05bb709880 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:23.911509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:23.923034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8757cd0-db0c-4f7a-802a-cd48c5f9ab21 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:24.015350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:24.016203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:24.016579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:24.017047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/domains/141a98759c7e470e8c4d0f19ecefb1cb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:24.019214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=141a98759c7e470e8c4d0f19ecefb1cb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:24.020956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:24.045258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.046000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.080841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.081585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.086550 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3619/7307] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:16:23 2022] PATCH /identity/v3/domains/141a98759c7e470e8c4d0f19ecefb1cb => generated 276 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:24.090058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:24.091759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:24.110055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ad941-0baa-4c37-85dc-d2ee78b1ed6f tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:24.196450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06228f6dd02641ebb8931b76e8e248b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '616dafbd4a014451ab951bf0eba31d7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:24.197088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:24.197407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:24.197634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] PATH_INFO: `/v3/domains/141a98759c7e470e8c4d0f19ecefb1cb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:24.210071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=141a98759c7e470e8c4d0f19ecefb1cb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:24.212095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:24.240791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.245006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.333694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.341657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.371676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.397544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] The token cache is being invalidate because project 141a98759c7e470e8c4d0f19ecefb1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:24.397895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.414111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.439578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.444884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.449977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-43a6dbda-dd13-4b85-ab93-d72c728155d2 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '141a98759c7e470e8c4d0f19ecefb1cb', 'request_id': 'req-43a6dbda-dd13-4b85-ab93-d72c728155d2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.459577 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3689/7308] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:24 2022] DELETE /identity/v3/domains/141a98759c7e470e8c4d0f19ecefb1cb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:24.463614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:24.465530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:24.542254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:24.547666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574cc0d-69ce-46ab-9fdd-241ecc2fc661 tempest-UsersV3TestJSON-2143128599 tempest-UsersV3TestJSON-2143128599-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:24.637686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:24.638442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:24.638819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:24.639289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] PATH_INFO: `/v3/users/4d0506932e8a4df5a5e30539840cd300` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:24.640949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0506932e8a4df5a5e30539840cd300)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:24.643059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:24.734937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0506932e8a4df5a5e30539840cd300'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.735677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0506932e8a4df5a5e30539840cd300'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.747311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0506932e8a4df5a5e30539840cd300'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.760470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0506932e8a4df5a5e30539840cd300'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.761460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0506932e8a4df5a5e30539840cd300'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:24.791423 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3620/7309] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:24 2022] DELETE /identity/v3/users/4d0506932e8a4df5a5e30539840cd300 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:24.793106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:24.793586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:24.793684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:24.809261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:24.817137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:24.818417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:24.899501 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3690/7310] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:24.911583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:24.914177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:24.997917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38af5e4-db9f-4767-991d-c49b3108962c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:25.003125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:25.004006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:25.004006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:25.004278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:25.017991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:25.019494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:25.035900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3621/7311] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:25.815343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:25.817814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:25.850750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aafc59-bf7a-445d-99a5-25b934056380 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:25.858646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:25.859403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:25.859688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:25.859959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] PATH_INFO: `/v3/projects/f147d87eb6624ba99b1875c87d127011` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:25.861290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f147d87eb6624ba99b1875c87d127011)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:25.863440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:25.947928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f147d87eb6624ba99b1875c87d127011', 'request_id': 'req-951da32a-adcb-4a89-9eaf-047e1ca4ba90'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:25.956722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f147d87eb6624ba99b1875c87d127011', 'request_id': 'req-951da32a-adcb-4a89-9eaf-047e1ca4ba90'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:25.964909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f147d87eb6624ba99b1875c87d127011', 'request_id': 'req-951da32a-adcb-4a89-9eaf-047e1ca4ba90'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:25.986958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] The token cache is being invalidate because project f147d87eb6624ba99b1875c87d127011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:25.987252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:25.987939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:25.989662 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3691/7312] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:25 2022] DELETE /identity/v3/projects/f147d87eb6624ba99b1875c87d127011 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:25.996444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:26.008599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badbdd0-114e-48cd-8071-49aeb3c99b99 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:26.057693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:26.120377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:26.121209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:26.121589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:26.121967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] PATH_INFO: `/v3/users/06228f6dd02641ebb8931b76e8e248b0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:26.123371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06228f6dd02641ebb8931b76e8e248b0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:26.125564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:26.191058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06228f6dd02641ebb8931b76e8e248b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.191626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06228f6dd02641ebb8931b76e8e248b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.200552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06228f6dd02641ebb8931b76e8e248b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.208374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06228f6dd02641ebb8931b76e8e248b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.209145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-942d53f1-0ab8-42e3-9a47-9490ff760a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06228f6dd02641ebb8931b76e8e248b0'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.248108 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3622/7313] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:25 2022] DELETE /identity/v3/users/06228f6dd02641ebb8931b76e8e248b0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:26.767304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:26.768927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:26.784214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951da32a-adcb-4a89-9eaf-047e1ca4ba90 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:26.849935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:26.850662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:26.850879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:26.851272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] PATH_INFO: `/v3/projects/616dafbd4a014451ab951bf0eba31d7f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:26.852404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616dafbd4a014451ab951bf0eba31d7f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:26.854314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:26.891301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:26.905740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616dafbd4a014451ab951bf0eba31d7f', 'request_id': 'req-a035aabb-75e9-4876-adc4-2270338c0b73'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.911497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616dafbd4a014451ab951bf0eba31d7f', 'request_id': 'req-a035aabb-75e9-4876-adc4-2270338c0b73'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.919365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616dafbd4a014451ab951bf0eba31d7f', 'request_id': 'req-a035aabb-75e9-4876-adc4-2270338c0b73'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.934065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] The token cache is being invalidate because project 616dafbd4a014451ab951bf0eba31d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:26.934360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.935073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a035aabb-75e9-4876-adc4-2270338c0b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:26.936734 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3692/7314] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:26 2022] DELETE /identity/v3/projects/616dafbd4a014451ab951bf0eba31d7f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:26.941726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:26.942006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:26.942277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:26.964773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:26.966008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:27.034566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3623/7315] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:27.042705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.043058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.043339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.067148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:27.069102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:27.081194 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3693/7316] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:27.082321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.095650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1e46-f9b1-4884-b3e9-579ce8bc27d1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.103849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.104462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.104731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.105191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.106685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:27.106685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.108294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.138341 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3624/7317] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:27.143838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.154993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb7657-7452-415c-b19d-cfc03408ec10 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.160172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.160912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.161258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.161716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.162550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.164819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.200776 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3694/7318] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:27.201586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.213673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30548b5c-e895-4c54-8f9e-7335db6e8c57 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.265428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:27.310774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.311420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.311722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.312090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.312720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.314679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.349451 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3625/7319] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:27.351599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.363059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaad42-1b01-411f-b292-e7ab64c34cce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.369611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.370436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.370684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.371026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.371843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.373895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.389434 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3695/7320] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:27.393935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.405094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f183f6f-d021-49e6-9a99-bc742027b2f9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.409887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.410519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.410827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.411094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.411741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.413458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2168656-7973-4853-ac59-c5417d7af4cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.426677 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3626/7321] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:27.429611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.441827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa506e-af54-46f4-996a-4ac200cd7488 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.447459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.448154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.448465 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.448741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] PATH_INFO: `/v3/projects/ca6f70c89e1242b7a50dd97eb863520e/users/d27c69318903468990a9ecdf73e37111/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.464202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6f70c89e1242b7a50dd97eb863520e, user_id=d27c69318903468990a9ecdf73e37111, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.466804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.481410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3696/7322] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:27 2022] PUT /identity/v3/projects/ca6f70c89e1242b7a50dd97eb863520e/users/d27c69318903468990a9ecdf73e37111/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:27.493364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.493698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.493976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.522645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] MFA Rules not processed for user `d27c69318903468990a9ecdf73e37111`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:27.523878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:27.594174 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3627/7323] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:27.599997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.682952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb461355-f091-4e2d-a07f-f84d2741e4ba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.688344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.688955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.689676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.690299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.691088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.692882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.728881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3697/7324] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/projects => generated 395 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:27.732925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.744485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db898c2d-6951-4f76-b7fe-f3f4e3b7ea89 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.810358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.811006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.811285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.811556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.812198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.814179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.838420 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3628/7325] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:27 2022] POST /identity/v3/users => generated 304 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:27.842353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.853631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793c5f3-1a3d-436e-aa94-aa88fde359a3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.858793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.859393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.859672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.859942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.860572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.862152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.877247 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3698/7326] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:27.878603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.888894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477b2b5-a801-49f6-a3bb-3383395f9ff4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.893630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:27.894279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:27.894602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:27.894859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] PATH_INFO: `/v3/projects/4c35a650e7cb418bb84e019ccb691f20/users/2d0ef452dba044ecbf315d632df46c88/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:27.911891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c35a650e7cb418bb84e019ccb691f20, user_id=2d0ef452dba044ecbf315d632df46c88, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:27.915518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:27.929660 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3629/7327] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:27 2022] PUT /identity/v3/projects/4c35a650e7cb418bb84e019ccb691f20/users/2d0ef452dba044ecbf315d632df46c88/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:27.933678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:27.948318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d782790-e875-4f6e-8aa6-2930842f02f8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:27.993701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:28.029057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.029769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.030130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.030479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.031224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.033552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f0f4c-ab34-4c06-aac4-3b4911283130 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.046983 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3699/7328] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:28.051014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.062890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b92bf9-6833-4fcf-871a-c4c20ed33a69 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.067911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.068742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.069085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.070104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] PATH_INFO: `/v3/projects/4c35a650e7cb418bb84e019ccb691f20/users/2d0ef452dba044ecbf315d632df46c88/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.071949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c35a650e7cb418bb84e019ccb691f20, user_id=2d0ef452dba044ecbf315d632df46c88, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.074930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.077410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:28.088589 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3630/7329] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:28 2022] PUT /identity/v3/projects/4c35a650e7cb418bb84e019ccb691f20/users/2d0ef452dba044ecbf315d632df46c88/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:28.098698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.099143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.099736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.134507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] MFA Rules not processed for user `2d0ef452dba044ecbf315d632df46c88`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:28.135921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:28.229060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3700/7330] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:28.232908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.313198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f0a69-bd60-4c58-81b5-b0e65482ad43 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.320947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c69318903468990a9ecdf73e37111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca6f70c89e1242b7a50dd97eb863520e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.321956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.322405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.322880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.324291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post Sep 24 21:16:28.329976 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call( Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. Sep 24 21:16:28.330966 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.336260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3631/7331] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/projects => generated 136 bytes in 104 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:16:28.337197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.350042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503799a-5852-4527-bba5-82703b9b7ffb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.356360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.357122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.357425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.357705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.358350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.360270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invalid input for field 'name': '' is too short Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: On instance['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: On instance['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: '' Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise error Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application '' Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.367960 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:16:28.369808 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application '' Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.371012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3701/7332] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/projects => generated 283 bytes in 33 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 24 21:16:28.375919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.390340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0b95a-2173-43f0-bae1-770d6c247edd tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.395378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.396172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.396493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.397049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.397852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.401772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: On instance['name']: Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: On instance['name']: Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise error Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:16:28.438111 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.439521 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application On instance['name']: Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 24 21:16:28.440594 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.441326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.441490 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3632/7333] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/projects => generated 412 bytes in 64 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) Sep 24 21:16:28.458181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2903a53-447d-4865-a7ba-0f4f7d380e6c tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.473745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.476395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.477127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.477610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects/450b575ca79442d99b2ba46b6a9983e5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.498081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:delete_project(project_id=450b575ca79442d99b2ba46b6a9983e5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.501787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Could not find project: 450b575ca79442d99b2ba46b6a9983e5.: keystone.exception.ProjectNotFound: Could not find project: 450b575ca79442d99b2ba46b6a9983e5. Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete Sep 24 21:16:28.520922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 450b575ca79442d99b2ba46b6a9983e5. Sep 24 21:16:28.522528 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:28.533745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.540570 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3702/7334] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:28 2022] DELETE /identity/v3/projects/450b575ca79442d99b2ba46b6a9983e5 => generated 113 bytes in 104 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:28.544757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd9f0a-ca7f-4073-9d07-83ed66cb2adf tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.549129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd27c69318903468990a9ecdf73e37111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca6f70c89e1242b7a50dd97eb863520e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.549793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.550142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.550480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.551212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] RBAC: Authorizing `identity:list_projects()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get Sep 24 21:16:28.557348 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._list_projects() Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application enforcer_obj._enforce( Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._enforcer.enforce( Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. Sep 24 21:16:28.558267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.564488 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3633/7335] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:16:28 2022] GET /identity/v3/projects => generated 135 bytes in 27 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:28.565668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.576744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b8b72-c9e2-4df0-9bab-ddac4c485ed0 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.582105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.582105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.582349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.582633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.583255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.585548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.631012 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3703/7336] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/projects => generated 358 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:28.640181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.656735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614904cc-4dd7-4132-b8b1-b4e7dacec0a6 tempest-ProjectsNegativeStaticTestJSON-1172661821 tempest-ProjectsNegativeStaticTestJSON-1172661821-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.726982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.727928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.727928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.728287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.728917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.730826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.752730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.sql.core [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-543326941' for key 'project.ixu_project_name_domain_id'") Sep 24 21:16:28.752730 ncdv-master-1 devstack@keystone.service[99133]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 21:16:28.752730 ncdv-master-1 devstack@keystone.service[99133]: [parameters: {'id': 'd3c675c4ca0a498f8e86e2d6a130d1b3', 'name': 'tempest-project-dup-543326941', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 21:16:28.752730 ncdv-master-1 devstack@keystone.service[99133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=99133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-543326941, project id d3c675c4ca0a498f8e86e2d6a130d1b3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-543326941, project id d3c675c4ca0a498f8e86e2d6a130d1b3. Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 21:16:28.772869 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-543326941' for key 'project.ixu_project_name_domain_id'") Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 21:16:28.773853 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application yield resource Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise self.value Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 21:16:28.774924 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application session.commit() Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.session.flush() Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application compat.raise_( Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 21:16:28.776357 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 21:16:28.777342 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application conn.query(q) Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application result.read() Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 21:16:28.778328 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-543326941' for key 'project.ixu_project_name_domain_id'") Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [parameters: {'id': 'd3c675c4ca0a498f8e86e2d6a130d1b3', 'name': 'tempest-project-dup-543326941', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.781665 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-543326941 at domain ID default. Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:28.782806 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-543326941, project id d3c675c4ca0a498f8e86e2d6a130d1b3. Sep 24 21:16:28.784008 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:28.786914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.787079 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3634/7337] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:16:28 2022] POST /identity/v3/projects => generated 284 bytes in 147 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:16:28.800353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8ea5f-9b93-4cea-8e60-bb18742c6193 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:28.807307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d0ef452dba044ecbf315d632df46c88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c35a650e7cb418bb84e019ccb691f20', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:28.809063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:28.809643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:28.810065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] PATH_INFO: `/v3/projects/cb0fe0852ffd48d299827ca1f85fadaf` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:28.811434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cb0fe0852ffd48d299827ca1f85fadaf)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:28.832447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:28.899133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0fe0852ffd48d299827ca1f85fadaf', 'request_id': 'req-59a632ed-e034-4f61-b637-1448c666b4f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:28.907491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0fe0852ffd48d299827ca1f85fadaf', 'request_id': 'req-59a632ed-e034-4f61-b637-1448c666b4f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:28.915056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0fe0852ffd48d299827ca1f85fadaf', 'request_id': 'req-59a632ed-e034-4f61-b637-1448c666b4f6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:28.919754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:28.933288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] The token cache is being invalidate because project cb0fe0852ffd48d299827ca1f85fadaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:28.934378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:28.935860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-59a632ed-e034-4f61-b637-1448c666b4f6 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:28.938008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3704/7338] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:28 2022] DELETE /identity/v3/projects/cb0fe0852ffd48d299827ca1f85fadaf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:28.942867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:28.945022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:29.016587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696618b5-23ea-4736-a87f-1ac6cd5211c9 tempest-ProjectsNegativeStaticTestJSON-230393284 tempest-ProjectsNegativeStaticTestJSON-230393284-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:29.048961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:29.103724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:29.105646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:29.106357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:29.107060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] PATH_INFO: `/v3/users/d27c69318903468990a9ecdf73e37111` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:29.108531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27c69318903468990a9ecdf73e37111)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:29.112413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:29.118228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ca5a-4cc2-4580-8355-6607319b0be5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:29.118228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ca5a-4cc2-4580-8355-6607319b0be5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:29.118228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869ca5a-4cc2-4580-8355-6607319b0be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:29.139925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2869ca5a-4cc2-4580-8355-6607319b0be5 None None] MFA Rules not processed for user `992bb56657f340919192b7624025e253`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:29.142072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2869ca5a-4cc2-4580-8355-6607319b0be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:29.192358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27c69318903468990a9ecdf73e37111'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:29.193710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27c69318903468990a9ecdf73e37111'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:29.203266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27c69318903468990a9ecdf73e37111'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:29.212232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27c69318903468990a9ecdf73e37111'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:29.213108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27c69318903468990a9ecdf73e37111'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:29.216782 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3705/7339] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:29.241617 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3635/7340] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:28 2022] DELETE /identity/v3/users/d27c69318903468990a9ecdf73e37111 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:29.243389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:29.243775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:29.244143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:29.267559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:29.269024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:29.352438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3706/7341] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:29.361692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:29.363219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:29.469922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5da70-f373-4936-b342-6b9c016705f0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:29.483386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:29.484004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:29.484284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:29.484558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:29.515505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:29.517173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:29.534724 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3636/7342] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:30.565774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:30.568199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:30.585821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133fb4-efa6-4c23-bc36-b1e704ed8245 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:30.590974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:30.591628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:30.591914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:30.592260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] PATH_INFO: `/v3/projects/ca6f70c89e1242b7a50dd97eb863520e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:30.593413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6f70c89e1242b7a50dd97eb863520e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:30.595382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:30.670370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6f70c89e1242b7a50dd97eb863520e', 'request_id': 'req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.678015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6f70c89e1242b7a50dd97eb863520e', 'request_id': 'req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.686752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6f70c89e1242b7a50dd97eb863520e', 'request_id': 'req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.707066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] The token cache is being invalidate because project ca6f70c89e1242b7a50dd97eb863520e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:30.707375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.708132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.711143 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3707/7343] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:30 2022] DELETE /identity/v3/projects/ca6f70c89e1242b7a50dd97eb863520e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:30.715794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:30.728493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7beb56-d164-4e4e-b834-ca815c202203 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:30.785079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:30.798173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:30.798862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:30.799142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:30.799423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] PATH_INFO: `/v3/users/2d0ef452dba044ecbf315d632df46c88` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:30.800570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0ef452dba044ecbf315d632df46c88)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:30.803183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:30.851042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0ef452dba044ecbf315d632df46c88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.851704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0ef452dba044ecbf315d632df46c88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.860144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0ef452dba044ecbf315d632df46c88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.867912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0ef452dba044ecbf315d632df46c88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.868582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-808368e1-b567-40e6-a28a-7bb25887cd3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0ef452dba044ecbf315d632df46c88'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:30.886705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3637/7344] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:30 2022] DELETE /identity/v3/users/2d0ef452dba044ecbf315d632df46c88 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:31.172237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.173819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:31.190611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac1f0a-a8aa-41cc-b05f-b36547d6fd8f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.263833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:31.335585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.348894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.349765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.350531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] PATH_INFO: `/v3/projects/4c35a650e7cb418bb84e019ccb691f20` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.352727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c35a650e7cb418bb84e019ccb691f20)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.364060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.429798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c35a650e7cb418bb84e019ccb691f20', 'request_id': 'req-f92c3262-9009-4ac5-94e1-3fc8251096fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:31.440036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c35a650e7cb418bb84e019ccb691f20', 'request_id': 'req-f92c3262-9009-4ac5-94e1-3fc8251096fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:31.458200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c35a650e7cb418bb84e019ccb691f20', 'request_id': 'req-f92c3262-9009-4ac5-94e1-3fc8251096fe'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:31.486256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] The token cache is being invalidate because project 4c35a650e7cb418bb84e019ccb691f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:31.486256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:31.486935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f92c3262-9009-4ac5-94e1-3fc8251096fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:31.488791 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3708/7345] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:31 2022] DELETE /identity/v3/projects/4c35a650e7cb418bb84e019ccb691f20 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:31.494128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.494412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.495348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.528138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:31.529899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:31.608178 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3638/7346] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:31.616665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.617143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.617294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.638845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:31.640064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:31.648078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3709/7347] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:31.652583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.662552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6989da-d78b-45c4-a6ab-e5e63927444d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.667152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.667749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.668032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.668307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.668958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:31.669262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.670953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.686023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3639/7348] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:31.690114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.699941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ec831-d6a0-4805-887a-b6ecc54004aa None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.704634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.705218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.705495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.705766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.706387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.708361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.735246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3710/7349] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:31 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:31.739564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.752557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e191ef-5cdd-47aa-aa42-1fd1f741224f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.806693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:31.814293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.815135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.815702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.816053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.816687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.819306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.841834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3640/7350] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:31 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:31.846033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.856831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ff24-04ef-4051-b9d1-a97c712b3ec6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.863317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.863573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.863792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.864062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.866053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.867420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.879125 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3711/7351] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:31.884468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.895475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118437-4a2c-48a8-a8b3-0d1c5f0f6bf0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.900964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.901935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.902195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.902441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.903100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.904725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f2953-73ef-498e-8cd5-0ae47f786a20 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.916391 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3641/7352] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:31.920521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:31.931837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a3da-fe76-4f8f-b86b-2dc818e19884 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:31.936574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:31.937140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.937417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.937688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] PATH_INFO: `/v3/projects/b99f66d25bbc41d681972633d25a2a86/users/5bc0116356b54ae1bf8b2d1e6a210add/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:31.954441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99f66d25bbc41d681972633d25a2a86, user_id=5bc0116356b54ae1bf8b2d1e6a210add, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:31.957281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:31.970974 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3712/7353] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:31 2022] PUT /identity/v3/projects/b99f66d25bbc41d681972633d25a2a86/users/5bc0116356b54ae1bf8b2d1e6a210add/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:31.982449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:31.982784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:31.983061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.014765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] MFA Rules not processed for user `5bc0116356b54ae1bf8b2d1e6a210add`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:32.016120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:32.085834 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3642/7354] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:32.090357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.142286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:32.157299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e794-60bd-4618-8027-9b577e96843d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.162832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.163552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.163841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.164118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.164764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.166712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.200780 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3713/7355] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:32.203214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.215383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bec0a-7540-418d-957c-a6d14dfc3d4b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.297639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.299543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.300012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.300454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.301634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.305396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.338928 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3643/7356] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:32.340124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.357890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d0632-6944-4ba8-9aa0-c14c7c645ce7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.371902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.372586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.372845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.373133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.374224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.377576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.402541 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3714/7357] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:32.404371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.415128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847382b2-7a19-4520-894c-7b805d67ec6a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.422368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.422368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.422368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.422368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] PATH_INFO: `/v3/projects/7224530f72004967936e777eab92894f/users/35572882ba4b46ffbd704e88a1330b2a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.447097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7224530f72004967936e777eab92894f, user_id=35572882ba4b46ffbd704e88a1330b2a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.449795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.467814 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3644/7358] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:32 2022] PUT /identity/v3/projects/7224530f72004967936e777eab92894f/users/35572882ba4b46ffbd704e88a1330b2a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:32.471764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.482794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a23fa-9308-4911-99d3-66b418e4d64f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.546235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.546890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.547169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.547590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.548238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.550057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4299e77-6375-4167-b897-a73d8375bdc7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.564617 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3715/7359] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:32.565976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.576855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338fba2-159d-4f17-9fa4-8a2a2e1b260c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.582112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.582813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.583092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.583971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] PATH_INFO: `/v3/projects/7224530f72004967936e777eab92894f/users/35572882ba4b46ffbd704e88a1330b2a/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.598790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7224530f72004967936e777eab92894f, user_id=35572882ba4b46ffbd704e88a1330b2a, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.598790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.615888 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3645/7360] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:32 2022] PUT /identity/v3/projects/7224530f72004967936e777eab92894f/users/35572882ba4b46ffbd704e88a1330b2a/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:32.625917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.626348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.626604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.669754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] MFA Rules not processed for user `35572882ba4b46ffbd704e88a1330b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:32.671087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:32.742259 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3716/7361] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:32.746643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.759254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d16e2-892a-4018-a631-1068db4ee93d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.764309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.765133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.765481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.765852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.766650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.768329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.786535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3646/7362] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/roles => generated 234 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:32.791147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.800542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a17209-e9e9-495e-8a31-4eea94136554 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.806548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.806548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.806788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.810385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.810385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.810385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.829032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3717/7363] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/roles => generated 235 bytes in 35 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:32.830560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.838343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:32.842591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e686ec1-8a3a-4332-a0e4-a570ca4df70c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.853840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.854551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.854839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.855109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.855788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.857444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.876418 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3647/7364] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/roles => generated 235 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:32.877879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.889025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27784bf-b4fc-45f7-9927-a314176b7910 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:32.893780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:32.894407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:32.894740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:32.897070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:32.897070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:32.898515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:32.917040 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3718/7365] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/domains => generated 277 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:32.920871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:32.936487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2b94c-e5f2-40aa-811f-57caf9cedcbf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.007739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.008298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.008581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.008857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.009491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.011639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.059990 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3648/7366] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:32 2022] POST /identity/v3/projects => generated 397 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:33.067324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.080025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b70b54-1f45-4067-8e9d-0e4b6986eecc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.098412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:33.145281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.146401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.146744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.147033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/groups` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.147693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.149637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.169038 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3719/7367] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:16:33 2022] POST /identity/v3/groups => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:33.171160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.185176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07890e2-0de6-4eb0-bfb6-535f63f65bfb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.190268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.190948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.191552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.192000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.192690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.194796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.218536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3649/7368] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:33 2022] POST /identity/v3/roles => generated 234 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:33.221069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.233811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cc942-5a21-43d5-9871-5aacd226dcdd tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.238936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.239563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.239848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.240123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.240931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.242925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.270093 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3720/7369] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:33 2022] POST /identity/v3/users => generated 442 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:33.271899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.283134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a0b86-296b-4db9-b880-73221936808d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.288740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.289361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.289648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.289928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.290625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.291670 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:33.291670 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:33.291670 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:33.292572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.312013 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3650/7370] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:33 2022] POST /identity/v3/roles => generated 269 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:33.316346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.327512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc2f08-83c1-4202-b43f-41c95b71b37d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.332489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.333169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.334156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.334623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/d7f89c65d82548699ed1fbfb2f32fa32` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.352427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=d7f89c65d82548699ed1fbfb2f32fa32)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.355298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.370925 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3721/7371] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:33 2022] PUT /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/d7f89c65d82548699ed1fbfb2f32fa32 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:33.375240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.390444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec6afb-abbb-4473-817a-570e75af070c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.453952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.454591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.454882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.455156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.455881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Adding query filter params (scope.project.id=f9ea008781a64493ae095b6895b153c1, user.id=ac0d4cd7bf0b42a296ab245f4b32967f) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:33.456194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.458062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.507593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:33.512410 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3651/7372] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:16:33 2022] GET /identity/v3/role_assignments?scope.project.id=f9ea008781a64493ae095b6895b153c1&user.id=ac0d4cd7bf0b42a296ab245f4b32967f&effective => generated 228 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:33.516471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.527304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff431-1d17-4016-a9f0-3f257c6b1ca1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.531962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.532616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.532902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.533176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/d7f89c65d82548699ed1fbfb2f32fa32` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.535477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=d7f89c65d82548699ed1fbfb2f32fa32)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.538925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.545881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role d7f89c65d82548699ed1fbfb2f32fa32 was removed from user ac0d4cd7bf0b42a296ab245f4b32967f on project f9ea008781a64493ae095b6895b153c1. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:33.546175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:33.546909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:33.562773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3722/7373] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:16:33 2022] DELETE /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/d7f89c65d82548699ed1fbfb2f32fa32 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:33.566961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.569089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:33.581710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b21ff-9872-4fc2-8d4c-8349f5b60975 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.659459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.660093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.660374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.660671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/d7f89c65d82548699ed1fbfb2f32fa32` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.662461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=d7f89c65d82548699ed1fbfb2f32fa32)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.663861 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:33.663861 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:33.663861 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:33.665408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.703864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role d7f89c65d82548699ed1fbfb2f32fa32 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:33.704165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:33.704835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:33.706922 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3652/7374] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:33 2022] DELETE /identity/v3/roles/d7f89c65d82548699ed1fbfb2f32fa32 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:33.712252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.722475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09444665-6a2c-4d66-bfea-f61986dfa183 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.796236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.796903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.797283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.797638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/35bc718f22774c1295e4adda89703c51` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.800092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=35bc718f22774c1295e4adda89703c51)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.803190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.816543 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3723/7375] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:33 2022] PUT /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/35bc718f22774c1295e4adda89703c51 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:33.821207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.822857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:33.837688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af1307-ba2b-4a0b-ab34-d064d4c2468a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:33.899263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:33.899963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:33.900302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:33.900577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:33.902484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:33.903525 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:33.903525 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:33.903525 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:33.904406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:33.947906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3653/7376] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:33 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 520 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:33.952281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:33.963638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836834-9e94-4f15-915c-b1e46a4ac683 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.015477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:34.030626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.031245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.031530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.031837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.032556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Adding query filter params (scope.project.id=f9ea008781a64493ae095b6895b153c1, user.id=ac0d4cd7bf0b42a296ab245f4b32967f) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:34.032866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.034927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.095853 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3724/7377] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:16:33 2022] GET /identity/v3/role_assignments?scope.project.id=f9ea008781a64493ae095b6895b153c1&user.id=ac0d4cd7bf0b42a296ab245f4b32967f&effective => generated 1061 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:34.100384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.113866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ada48-3eb4-4ead-898b-476a1a861e50 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.119835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.120770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.121124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.121518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.123246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.124326 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:34.124326 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:34.124326 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:34.125528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.138160 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3654/7378] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.142315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.152861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778ad6-2a21-454d-8669-631adb3c99bb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.219657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.220279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.220545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.220828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.221592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Adding query filter params (scope.project.id=f9ea008781a64493ae095b6895b153c1, user.id=ac0d4cd7bf0b42a296ab245f4b32967f) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:34.221878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.223755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.287384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3725/7379] 10.222.0.52 () {58 vars in 1356 bytes} [Sat Sep 24 21:16:34 2022] GET /identity/v3/role_assignments?scope.project.id=f9ea008781a64493ae095b6895b153c1&user.id=ac0d4cd7bf0b42a296ab245f4b32967f&effective => generated 577 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:34.291425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.301815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e8078-a199-4994-a41a-fb3f4cce289a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.307342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.307534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.307862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.308224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.310275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.311636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475.: keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = query.one() Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._iter().one() Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._only_one_row( Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:34.335409 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:34.336769 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:34.337935 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3655/7380] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 137 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:34.341580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.353067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9340a95-2ead-4c65-bc9f-0eb642f5fd36 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.357879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.358581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.358869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.359143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/35bc718f22774c1295e4adda89703c51` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.361656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=35bc718f22774c1295e4adda89703c51)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.369183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.375699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 35bc718f22774c1295e4adda89703c51 was removed from user ac0d4cd7bf0b42a296ab245f4b32967f on project f9ea008781a64493ae095b6895b153c1. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:34.376182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.377081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.397564 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3726/7381] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/35bc718f22774c1295e4adda89703c51 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.403032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.404734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:34.418531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a00160-753a-4325-bd23-99293ea127a2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.448987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:34.486916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.487534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.487820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.488099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.488770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.491045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.506196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3656/7382] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:34 2022] POST /identity/v3/roles => generated 271 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:34.510684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.520621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675cb61-37a2-4048-afee-f1d442b8d654 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.525570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.526433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.526758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.527042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.527723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Adding query filter params (domain_id=8ada586d09934f36979aa3d2425aa45e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:34.528035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.529078 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:34.529078 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:34.529078 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:34.529933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.539553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3727/7383] 10.222.0.52 () {58 vars in 1207 bytes} [Sat Sep 24 21:16:34 2022] GET /identity/v3/roles?domain_id=8ada586d09934f36979aa3d2425aa45e => generated 408 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:34.543776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.554976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab32d-3c88-4006-9963-458904cfae7d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.560449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.561075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.561357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.561632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/2dc3773d0fc04ce48d6fb7b0043c30b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.562785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=2dc3773d0fc04ce48d6fb7b0043c30b6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.564414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.603775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 2dc3773d0fc04ce48d6fb7b0043c30b6 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:34.604067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.604737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.606746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3657/7384] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/roles/2dc3773d0fc04ce48d6fb7b0043c30b6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.610774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.612456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:34.626095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215a366-227c-4a99-a404-7f26c9b36375 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.663992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:34.693703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.694327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.694649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.694930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.695588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Adding query filter params (domain_id=8ada586d09934f36979aa3d2425aa45e) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:34.695891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.698017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.707470 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3728/7385] 10.222.0.52 () {58 vars in 1207 bytes} [Sat Sep 24 21:16:34 2022] GET /identity/v3/roles?domain_id=8ada586d09934f36979aa3d2425aa45e => generated 148 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:34.709745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.720083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe4724-7e82-4754-a8c7-d4fdb4a4b703 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.725647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.726284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.726626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.726899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/2dc3773d0fc04ce48d6fb7b0043c30b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.739464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Could not find role: 2dc3773d0fc04ce48d6fb7b0043c30b6.: keystone.exception.RoleNotFound: Could not find role: 2dc3773d0fc04ce48d6fb7b0043c30b6. Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete Sep 24 21:16:34.744471 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:16:34.745502 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2dc3773d0fc04ce48d6fb7b0043c30b6. Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:34.746668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3658/7386] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/roles/2dc3773d0fc04ce48d6fb7b0043c30b6 => generated 110 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:34.751866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.762633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adcfe0-cd74-4c01-90bc-43b232e967aa tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.767652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.768313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.772083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.772083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.785595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.789205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.807851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3729/7387] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:34 2022] PUT /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.811995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.823159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a696b05-3820-460f-b54a-5ff7abfba575 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.880708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.881275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.881555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.881830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.883521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=8ada586d09934f36979aa3d2425aa45e, group_id=25588bd2a95a44d7a99161ece1b137ac)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.884546 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:34.884546 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:34.884546 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:34.886348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.899099 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3659/7388] 10.222.0.52 () {58 vars in 1365 bytes} [Sat Sep 24 21:16:34 2022] GET /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles => generated 409 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:34.903548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.914661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b908000-a26e-4779-a117-36bc36f7c4c6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.919354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.920005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.920291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.920570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.922786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.925422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.931363 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3730/7389] 10.222.0.52 () {58 vars in 1465 bytes} [Sat Sep 24 21:16:34 2022] HEAD /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.935306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.945656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd1cd-8b90-46ef-9ec9-0387ffc0fd2a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:34.950457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:34.951142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:34.951719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:34.951719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:34.954096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:34.956802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:34.963406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 6348848c67964d5b904d4319333de23e was removed from group 25588bd2a95a44d7a99161ece1b137ac on domain 8ada586d09934f36979aa3d2425aa45e. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:34.963824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.964605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:34.982163 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3660/7390] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:16:34 2022] DELETE /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:34.987311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:34.989071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:35.003019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565839f9-056f-407c-8d4c-5b175ae33f12 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.062136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.062937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.063222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.063597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.065825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f9ea008781a64493ae095b6895b153c1, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.069078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.085029 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3731/7391] 10.222.0.52 () {60 vars in 1486 bytes} [Sat Sep 24 21:16:34 2022] PUT /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.089072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.100276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db74c6b-30b1-40f0-a36f-f2427d56edd6 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.139418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:35.159753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.160404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.160694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.160976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.163039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_grants(project_id=f9ea008781a64493ae095b6895b153c1, group_id=25588bd2a95a44d7a99161ece1b137ac)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.165662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182eaef5-41a0-43bf-8ca8-2b9765c28172 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.176890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3661/7392] 10.222.0.52 () {58 vars in 1368 bytes} [Sat Sep 24 21:16:35 2022] GET /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles => generated 410 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:35.181350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.191891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887b5ef-ea06-45ea-a466-96e70bddb316 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.197677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.198313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.198641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.198922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/groups/25588bd2a95a44d7a99161ece1b137ac/users/ac0d4cd7bf0b42a296ab245f4b32967f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.200628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=25588bd2a95a44d7a99161ece1b137ac, user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.202544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.227613 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3732/7393] 10.222.0.52 () {60 vars in 1360 bytes} [Sat Sep 24 21:16:35 2022] PUT /identity/v3/groups/25588bd2a95a44d7a99161ece1b137ac/users/ac0d4cd7bf0b42a296ab245f4b32967f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.232062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.232421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.232740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.273297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] MFA Rules not processed for user `ac0d4cd7bf0b42a296ab245f4b32967f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:35.274522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:35.372559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3662/7394] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:35.377000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.440050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660af968-bc55-4b19-aa0f-5e25aa0b88bc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.445151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.445928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.446207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.446488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.448833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_grant(project_id=f9ea008781a64493ae095b6895b153c1, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.451546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.457842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3733/7395] 10.222.0.52 () {58 vars in 1468 bytes} [Sat Sep 24 21:16:35 2022] HEAD /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.462791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.476665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70063056-419f-43b0-9609-3b667cd6e89c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.481612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.482268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.482580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.482914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.485043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f9ea008781a64493ae095b6895b153c1, group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.487747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.493627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 6348848c67964d5b904d4319333de23e was removed from group 25588bd2a95a44d7a99161ece1b137ac on project f9ea008781a64493ae095b6895b153c1. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:35.493916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:35.494611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:35.507327 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3663/7396] 10.222.0.52 () {58 vars in 1470 bytes} [Sat Sep 24 21:16:35 2022] DELETE /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.511543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.513255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:35.527025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95719-75ff-40f3-8bc5-594d6f45d8f1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.538989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:35.588758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.589401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.589682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.589957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/groups/25588bd2a95a44d7a99161ece1b137ac/users/ac0d4cd7bf0b42a296ab245f4b32967f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.591647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=25588bd2a95a44d7a99161ece1b137ac, user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.593571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.602923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:35.617278 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3734/7397] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:35 2022] DELETE /identity/v3/groups/25588bd2a95a44d7a99161ece1b137ac/users/ac0d4cd7bf0b42a296ab245f4b32967f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:35.622448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.633830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcf87e-2e91-4f90-a057-afc1ecc36d59 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.691202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.691771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.692052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.692326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.694190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.695254 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:35.695254 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:35.695254 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:35.696181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.702983 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3664/7398] 10.222.0.52 () {60 vars in 1381 bytes} [Sat Sep 24 21:16:35 2022] PUT /identity/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.707039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.717921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8d79d-2aed-43ec-ae9d-f0782451ef0a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.722663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.723291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.723578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.723941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.725711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=25588bd2a95a44d7a99161ece1b137ac)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.726797 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:35.726797 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:35.726797 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:35.727652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.739376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3735/7399] 10.222.0.52 () {58 vars in 1263 bytes} [Sat Sep 24 21:16:35 2022] GET /identity/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles => generated 375 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:35.743589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.753627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9af57-d521-4bba-8196-44c4f48efbb3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.758893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.759517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.759798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.760074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.761723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.762754 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:35.762754 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:35.762754 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:35.763662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.767273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:35.771417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3665/7400] 10.222.0.52 () {58 vars in 1363 bytes} [Sat Sep 24 21:16:35 2022] HEAD /identity/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 27 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:35.775546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.786127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ba6be-9e82-4655-bb85-231cedee9d0b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.791549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.792216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.792502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.792938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.794646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=25588bd2a95a44d7a99161ece1b137ac, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.795661 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:35.795661 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:35.795661 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:35.796534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.805065 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3736/7401] 10.222.0.52 () {58 vars in 1365 bytes} [Sat Sep 24 21:16:35 2022] DELETE /identity/v3/system/groups/25588bd2a95a44d7a99161ece1b137ac/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.810085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.822301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279626f6-a742-45c6-b319-a2093cc9fd38 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.829414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.830085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.830356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.830668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.832901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.836597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.853026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3666/7402] 10.222.0.52 () {60 vars in 1480 bytes} [Sat Sep 24 21:16:35 2022] PUT /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:35.857154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.868394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e09b91-aa8a-4b13-8aac-401aa9eb5800 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.931000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.931771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.932049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.932321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.934024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=8ada586d09934f36979aa3d2425aa45e, user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.936748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.950379 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3737/7403] 10.222.0.52 () {58 vars in 1362 bytes} [Sat Sep 24 21:16:35 2022] GET /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles => generated 408 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:35.954475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.965880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ffc5d-d7fa-4bde-ae9b-f139d17d6d42 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:35.970756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:35.971427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:35.971735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:35.972013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:35.974295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:35.977022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:35.982962 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3667/7404] 10.222.0.52 () {58 vars in 1462 bytes} [Sat Sep 24 21:16:35 2022] HEAD /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:35.986986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:35.997226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27576-65de-4c7a-8ade-aff7c645ddeb tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.001821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.002457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.002778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.003052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.005890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=8ada586d09934f36979aa3d2425aa45e, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.008570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.013767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 6348848c67964d5b904d4319333de23e was removed from user ac0d4cd7bf0b42a296ab245f4b32967f on domain 8ada586d09934f36979aa3d2425aa45e. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:36.014051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:36.014739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:36.029537 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3738/7405] 10.222.0.52 () {58 vars in 1464 bytes} [Sat Sep 24 21:16:35 2022] DELETE /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.034798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.036653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:36.049496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c0ba6-c7e1-44ea-b643-b0c9cf17dec3 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.106939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.107589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.107874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.108152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.110339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.113026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.127998 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3668/7406] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.132980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.144079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cb99f-bef3-4c8f-afaf-8257657a4143 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.195564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:36.207205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.207796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.208087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.208424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.210179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_grants(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.212799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.227580 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3739/7407] 10.222.0.52 () {58 vars in 1365 bytes} [Sat Sep 24 21:16:36 2022] GET /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles => generated 409 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:36.228846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.241025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae8532-c8ef-4d01-b1d7-7a98122ebee1 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.245944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.246661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.246947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.247251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.249601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.252336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.259476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3669/7408] 10.222.0.52 () {58 vars in 1465 bytes} [Sat Sep 24 21:16:36 2022] HEAD /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.263624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.274346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bc0f4-785a-4b3f-b3e6-0ccdf630bd03 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.279521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.280177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.280460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.280735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.282962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=f9ea008781a64493ae095b6895b153c1, user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.285592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.291379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 6348848c67964d5b904d4319333de23e was removed from user ac0d4cd7bf0b42a296ab245f4b32967f on project f9ea008781a64493ae095b6895b153c1. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:36.291674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:36.292347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:36.306415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3740/7409] 10.222.0.52 () {58 vars in 1467 bytes} [Sat Sep 24 21:16:36 2022] DELETE /identity/v3/projects/f9ea008781a64493ae095b6895b153c1/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.311531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.313231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:36.327618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377069cd-0c43-4729-a39c-630e7e242993 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.389689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.390679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.390951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.391225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.393106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.396133 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.396133 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:36.396133 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:36.396133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.403062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3670/7410] 10.222.0.52 () {60 vars in 1378 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:36.407149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.417772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93358398-4114-42b1-af6e-a0fa7020f020 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.422723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.423354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.423634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.423906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.425051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.426089 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.426089 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:36.426089 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:36.427541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.441040 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3741/7411] 10.222.0.52 () {58 vars in 1260 bytes} [Sat Sep 24 21:16:36 2022] GET /identity/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles => generated 374 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:36.446293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.455972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b26fce-9c79-4484-8408-9f1608bd71f8 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.461155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.461732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.462041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.462321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.464046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.465051 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.465051 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:36.465051 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:36.465992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.471334 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3671/7412] 10.222.0.52 () {58 vars in 1360 bytes} [Sat Sep 24 21:16:36 2022] HEAD /identity/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 26 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.475577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.485528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696442ff-1df1-42c9-8862-3ff64783d519 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.491292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.491935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.492224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.492502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.494197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=ac0d4cd7bf0b42a296ab245f4b32967f, role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.495277 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.495277 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:36.495277 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:36.496160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.503246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3742/7413] 10.222.0.52 () {58 vars in 1362 bytes} [Sat Sep 24 21:16:36 2022] DELETE /identity/v3/system/users/ac0d4cd7bf0b42a296ab245f4b32967f/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 28 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:36.508269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.518313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c7625-490e-4610-83ca-68a8ea9febfc tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.523208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.523798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.524078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.524352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.525003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.526989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.541906 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3672/7414] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:36 2022] POST /identity/v3/roles => generated 269 bytes in 34 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.545905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.556535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98461f-fc9a-41a7-a8d3-b736b400999a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.561226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.561811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.562093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.562368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.563062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.564095 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.564095 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:36.564095 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:36.564977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.584622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3743/7415] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:36 2022] POST /identity/v3/roles => generated 270 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:36.585768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.598206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9cfe9-6646-464e-99ff-26291118ff58 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.605079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.605738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.606020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.606303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/4738143d2b594226988e8eea3e049419` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.608376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=4738143d2b594226988e8eea3e049419)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.610165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.643196 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3673/7416] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/4738143d2b594226988e8eea3e049419 => generated 530 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.647439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.657996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20f79a-4840-433d-84b7-222e7ee00b89 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.715901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.716423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.716707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.716980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.717642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.719280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.738797 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3744/7417] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:16:36 2022] POST /identity/v3/domains => generated 277 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.740850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.751603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0633277-8fd5-4995-87cd-638bfab71992 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.761574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:36.812545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.813157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.813470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.813830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.814443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.816118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.831417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3674/7418] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:36 2022] POST /identity/v3/roles => generated 269 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.836253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.846665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d02e9f-1e8b-498f-afab-ec013f683fda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.852169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.852811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.853093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.853367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/ad616b10d230480f94a63d29a14d920e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.855046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=ad616b10d230480f94a63d29a14d920e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.856075 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:36.856075 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:36.856075 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:36.856943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.886174 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3745/7419] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/ad616b10d230480f94a63d29a14d920e => generated 529 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.890266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:36.901742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cfc7-ae5a-49f9-917c-4265af15d7ab tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:36.957971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:36.959181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:36.959460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:36.959789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:36.961456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:36.963104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:36.990682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3675/7420] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/6348848c67964d5b904d4319333de23e => generated 524 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:36.994966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.005691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff9f3d-3529-4f09-ac0e-0f9d63099a73 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.062306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:37.072992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.074442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.075062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.075615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/6348848c67964d5b904d4319333de23e/implies/8b64b81267ee4a599d59b62553fe6eff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.078369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=6348848c67964d5b904d4319333de23e, implied_role_id=8b64b81267ee4a599d59b62553fe6eff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.081877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] 8b64b81267ee4a599d59b62553fe6eff cannot be an implied roles.: keystone.exception.InvalidImpliedRole: 8b64b81267ee4a599d59b62553fe6eff cannot be an implied roles. Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put Sep 24 21:16:37.102312 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: 8b64b81267ee4a599d59b62553fe6eff cannot be an implied roles. Sep 24 21:16:37.103551 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:37.104940 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3746/7421] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:36 2022] PUT /identity/v3/roles/6348848c67964d5b904d4319333de23e/implies/8b64b81267ee4a599d59b62553fe6eff => generated 116 bytes in 110 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:37.109103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.121387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d84c-5c2f-43be-b703-7e08f8360a53 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.127388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.128125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.128508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.128900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.131161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.132949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.146746 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3676/7422] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/6348848c67964d5b904d4319333de23e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:37.148861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.161452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6463f6-8f2f-4085-9ce4-0718165a357d tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.226142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.227191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.227487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.227875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/ad616b10d230480f94a63d29a14d920e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.229771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=ad616b10d230480f94a63d29a14d920e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.230768 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:37.230768 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:37.230768 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:37.231656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.247473 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3747/7423] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/ad616b10d230480f94a63d29a14d920e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:37.251659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.263009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0557e-4536-4767-b304-47ad775ef588 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.326884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.327942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.328303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.328578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/ad616b10d230480f94a63d29a14d920e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.329745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=ad616b10d230480f94a63d29a14d920e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.331390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.350604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:37.385983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role ad616b10d230480f94a63d29a14d920e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:37.386842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.387659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.390172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3677/7424] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/roles/ad616b10d230480f94a63d29a14d920e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:37.394678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.397935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:37.416323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3752-5add-4b59-829a-ff54266ee0ce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.502645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.503173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.503461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.503787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/6630965ee3b848fe803a0a66fab9f0d3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.525087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6630965ee3b848fe803a0a66fab9f0d3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.527413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.552105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.553521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.585683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.586324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.592039 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3748/7425] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:16:37 2022] PATCH /identity/v3/domains/6630965ee3b848fe803a0a66fab9f0d3 => generated 278 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:37.593278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.606166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86339-db67-4594-9046-9f8d1cf47a02 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.674363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.675053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.675334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.675610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/6630965ee3b848fe803a0a66fab9f0d3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.687947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6630965ee3b848fe803a0a66fab9f0d3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.689581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.719064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.731770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.820063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.828433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.838567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.853525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] The token cache is being invalidate because project 6630965ee3b848fe803a0a66fab9f0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:37.853906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.854726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.864229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.865142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.869673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6630965ee3b848fe803a0a66fab9f0d3', 'request_id': 'req-7a48554e-7e2a-4935-9eee-7483bee3515f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:37.876097 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3678/7426] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/domains/6630965ee3b848fe803a0a66fab9f0d3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:37.880413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.881834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:37.894920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ae151-4152-4a88-8a90-f009a63b418c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:37.934990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:37.965330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:37.965978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:37.966260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:37.966569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/4738143d2b594226988e8eea3e049419` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:37.968391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=8b64b81267ee4a599d59b62553fe6eff, implied_role_id=4738143d2b594226988e8eea3e049419)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:37.970525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:37.980288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3749/7427] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff/implies/4738143d2b594226988e8eea3e049419 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:37.984773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:37.986372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:38.003095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48554e-7e2a-4935-9eee-7483bee3515f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.025269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:38.076291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.077003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.077299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.077610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/4738143d2b594226988e8eea3e049419` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.078902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=4738143d2b594226988e8eea3e049419)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.081032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.119543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 4738143d2b594226988e8eea3e049419 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:38.119813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:38.120473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:38.122298 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3679/7428] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:37 2022] DELETE /identity/v3/roles/4738143d2b594226988e8eea3e049419 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:38.126673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.128403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:38.142053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121ff1-bbbc-40b4-ae28-1bc4742fa670 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.201305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.201946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.202343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.202664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/8b64b81267ee4a599d59b62553fe6eff` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.203814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=8b64b81267ee4a599d59b62553fe6eff)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.204858 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:38.204858 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:38.204858 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:38.205741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.238839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 8b64b81267ee4a599d59b62553fe6eff has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:38.239128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:38.240002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:38.241767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3750/7429] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:38 2022] DELETE /identity/v3/roles/8b64b81267ee4a599d59b62553fe6eff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:38.247182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.258145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf0294-2c8e-4352-abe2-8180c13457f4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.316849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.318121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.318630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.319041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.320733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.322323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.352959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3680/7430] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:38 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 520 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:38.354148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.355560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:38.369267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06625ff6-9f13-4ab9-b9e5-4fbec529bb9a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.434973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.436186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `HEAD` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.436300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.436579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.438212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.439491 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:38.439491 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:38.439491 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:38.440385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.441293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.442173 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:38.442173 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:38.442173 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:38.443138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.449010 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3751/7431] 10.222.0.52 () {58 vars in 1345 bytes} [Sat Sep 24 21:16:38 2022] HEAD /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:38.453369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.464527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e483fe-e563-4bea-9c11-ce708e4dd4cf tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.469410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.470056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.470339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.470654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.472285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.473309 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:38.473309 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:38.473309 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:38.475916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.483901 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3681/7432] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:38 2022] GET /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 520 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:38.485042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.495164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16075d-2f24-493a-8fa8-d40cbea2fa75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.500651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.501296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.501650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.501938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.503658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.505339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.514066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3752/7433] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:38 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:38.518800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.531313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30677-9dba-4a9e-a425-1b20ec6d3635 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.593135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:38.607420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.608770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.609143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.609709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.612423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.614039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475.: keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = query.one() Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._iter().one() Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._only_one_row( Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:38.623380 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 24 21:16:38.624651 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 24 21:16:38.625571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 24 21:16:38.625571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:38.625571 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:16:38.625571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3682/7434] 10.222.0.52 () {58 vars in 1344 bytes} [Sat Sep 24 21:16:38 2022] GET /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 137 bytes in 106 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:16:38.628753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.641900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370b2ae-76f7-4f4e-815f-89bef06ce75c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.648156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.648936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.649222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.649494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.651404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.652974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475.: keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = query.one() Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2869, in one Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._iter().one() Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1476, in one Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._only_one_row( Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exc.NoResultFound( Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:38.662943 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) Sep 24 21:16:38.663996 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 35bc718f22774c1295e4adda89703c51 does not imply f1478b1a842f4378bbfe2cd5d9c9f475. Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:38.665378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3753/7435] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:38 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 137 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:16:38.669906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.680821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01d44a-2a49-4469-b7d3-25ffd5c769c4 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.685734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.686182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.686472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.686777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.688406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.690009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.720023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3683/7436] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:38 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 520 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:38.724472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.737660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454b4f4-9241-4129-b93e-4d4e4bf1ad25 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.752464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:38.800704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.801359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.801645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.802653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.804439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.806052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.845047 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3754/7437] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:38 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88 => generated 520 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:38.867410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.878039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e4282-196e-4750-8224-d6b0e07e0854 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:38.938120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:38.938809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:38.939086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:38.939360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:38.940999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=f7b76de8f7df4539a273afee68d69d88, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:38.943250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:38.968794 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3684/7438] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:38 2022] PUT /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e => generated 520 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:38.973018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:38.985002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79a130-925e-45f6-865f-378a82211132 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.045222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.045840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.046125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.046405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.047000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.048027 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:39.048027 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:39.048027 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:39.048919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.067108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3755/7439] 10.222.0.52 () {58 vars in 1152 bytes} [Sat Sep 24 21:16:38 2022] GET /identity/v3/role_inferences => generated 1606 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:39.072205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.084845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1b90-145a-41d4-a27e-6f3f5907e392 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.098517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.098517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.098517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.099078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.099078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=f7b76de8f7df4539a273afee68d69d88, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.099078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.106032 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3685/7440] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:39 2022] DELETE /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:39.109855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.123293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811f89b-aabd-44cc-b328-43ec35174141 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.188141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.189276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.189276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.189578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.191363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.193734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.205850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3756/7441] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:39 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:39.207118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.219119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6120c5-4930-4d0e-bb9c-24bf43bb74a5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.291633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.292398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.292791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.293226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.296172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.297915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.308522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3686/7442] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:39 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:39.313454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.324748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915b6d7-7fc2-4146-bf9d-43b54bfc3210 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.354151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:39.391590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.392209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.392492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.392769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.393417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.395116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.410552 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3757/7443] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:39 2022] GET /identity/v3/roles => generated 2322 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:39.414665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.425751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4247955-d127-4ec5-ad42-db3101e4d51b tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.431399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.431518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.431757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.432066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.432740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.434380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.449162 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3687/7444] 10.222.0.52 () {60 vars in 1143 bytes} [Sat Sep 24 21:16:39 2022] POST /identity/v3/roles => generated 235 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:39.453336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.468140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c1fec-90c9-425b-a6de-1119bf2ade7f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.472977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.473608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.474547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.474867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/69c746c0b6544afbb88f718ddc475011` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.476315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:update_role(role_id=69c746c0b6544afbb88f718ddc475011)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.477324 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:39.477324 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:39.477324 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:39.478198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.509529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3758/7445] 10.222.0.52 () {60 vars in 1243 bytes} [Sat Sep 24 21:16:39 2022] PATCH /identity/v3/roles/69c746c0b6544afbb88f718ddc475011 => generated 237 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:39.513509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.524420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bdcac-2c07-4204-aca5-75bb1a30c57a tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.529378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.530022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.530337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.530665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/69c746c0b6544afbb88f718ddc475011` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.539581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:get_role(role_id=69c746c0b6544afbb88f718ddc475011)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.541286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.542609 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3688/7446] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:16:39 2022] GET /identity/v3/roles/69c746c0b6544afbb88f718ddc475011 => generated 237 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:39.546313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.556546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38c7d9-9a8e-4272-91d2-2def125a894f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.561105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.561689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.561967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.562236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.562894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.564469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.576565 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3759/7447] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:39 2022] GET /identity/v3/roles => generated 2550 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:39.580500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.591116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4b33-1029-4933-8403-f4b2342c2919 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.595789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.596788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.597151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.597489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/69c746c0b6544afbb88f718ddc475011` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.599553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_role(role_id=69c746c0b6544afbb88f718ddc475011)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.601781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.609079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:39.640892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 69c746c0b6544afbb88f718ddc475011 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:39.641178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:39.641952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:39.643800 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3689/7448] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:39 2022] DELETE /identity/v3/roles/69c746c0b6544afbb88f718ddc475011 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:39.649836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.651841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:39.664937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ac45d-df5f-47ae-aa10-b26f2b77ecf2 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.720316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.720947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.721226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.721500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.723135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.725655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.748938 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3760/7449] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:39 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 520 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:39.753272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.763867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd2852-fa12-48b9-99cc-8fdfef93e3ea tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.824297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.824949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.825231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.825499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.827198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.828849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.854231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3690/7450] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:39 2022] PUT /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88 => generated 520 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:39.858240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.873415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0a92c-0e80-4f9b-ac8f-a846b5ef0656 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:39.933981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:39.935080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:39.935486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:39.935964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:39.937750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=f7b76de8f7df4539a273afee68d69d88, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:39.939377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:39.948276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:39.978632 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3761/7451] 10.222.0.52 () {60 vars in 1363 bytes} [Sat Sep 24 21:16:39 2022] PUT /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e => generated 520 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:39.982857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:39.994299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b89f4-f6d0-4979-a56b-748c54f04489 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.066230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.066896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.067174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.067490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.068835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.069857 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:40.069857 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:16:40.069857 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:16:40.070772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.076666 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3691/7452] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Sep 24 21:16:39 2022] GET /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies => generated 489 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:40.080912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.092174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed58a0-2187-4d03-b6f9-c214aa99cbce tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.096847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.097462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.097741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.098010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.099787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=35bc718f22774c1295e4adda89703c51)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.100784 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:16:40.100784 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:16:40.100784 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:16:40.101654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.111081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3762/7453] 10.222.0.52 () {58 vars in 1245 bytes} [Sat Sep 24 21:16:40 2022] GET /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies => generated 660 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:40.113914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.124746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be0d30-bc62-4a13-a454-bb668d87f030 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.129529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.132886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.132886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.132886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.135182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=f7b76de8f7df4539a273afee68d69d88, implied_role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.135182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.144445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3692/7454] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88/implies/6348848c67964d5b904d4319333de23e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.149496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.160027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7ade-b10b-4c27-8a52-1f71b716e83e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.227634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.228629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.228910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.229184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.230991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.232664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.242767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3763/7455] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f7b76de8f7df4539a273afee68d69d88 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.246990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.257994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a588fe-7209-4cec-b6ae-1809ca1fd01f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.292707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:40.317310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.318855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.319137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.319411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.321021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=35bc718f22774c1295e4adda89703c51, implied_role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.322654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.330306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3693/7456] 10.222.0.52 () {58 vars in 1347 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51/implies/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.336819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.347772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62666eba-94ed-4a91-a6c6-02d495532873 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.406373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.406957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.407240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.407513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/6630965ee3b848fe803a0a66fab9f0d3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.414288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Could not find domain: 6630965ee3b848fe803a0a66fab9f0d3.: keystone.exception.DomainNotFound: Could not find domain: 6630965ee3b848fe803a0a66fab9f0d3. Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 6630965ee3b848fe803a0a66fab9f0d3. Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:40.419254 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:40.420394 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 6630965ee3b848fe803a0a66fab9f0d3. Sep 24 21:16:40.424593 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:40.425705 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3764/7457] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:16:40 2022] PATCH /identity/v3/domains/6630965ee3b848fe803a0a66fab9f0d3 => generated 112 bytes in 84 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:16:40.425813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.435872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132780e-28f5-490a-b031-162781fec1ed tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.440719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.441337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.441616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.441915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/users/ac0d4cd7bf0b42a296ab245f4b32967f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.443660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ac0d4cd7bf0b42a296ab245f4b32967f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.445556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.489428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.490096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.497738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.505429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.506061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0d4cd7bf0b42a296ab245f4b32967f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.524739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3694/7458] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/users/ac0d4cd7bf0b42a296ab245f4b32967f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.529012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.530815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:40.548001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8edcce-c592-4c38-8f31-e19c1590234f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.617129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.618313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.618679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.619074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/6348848c67964d5b904d4319333de23e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.620560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_role(role_id=6348848c67964d5b904d4319333de23e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.622145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.642633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:40.666319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 6348848c67964d5b904d4319333de23e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:40.666557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.667242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.669010 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3765/7459] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/roles/6348848c67964d5b904d4319333de23e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.672988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.683702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccda7a-1a07-41ad-ab28-c934f333f49c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.741671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.742316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.742652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.742957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/groups/25588bd2a95a44d7a99161ece1b137ac` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.744157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_group(group_id=25588bd2a95a44d7a99161ece1b137ac)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.746082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.788446 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3695/7460] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/groups/25588bd2a95a44d7a99161ece1b137ac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.792455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.793852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:40.807280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fedce8-e086-45db-b567-b4c0a375c07e tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.862797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:40.863426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:40.863708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:40.863981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/projects/f9ea008781a64493ae095b6895b153c1` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:40.865635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f9ea008781a64493ae095b6895b153c1)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:40.867586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:40.922811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ea008781a64493ae095b6895b153c1', 'request_id': 'req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.930070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ea008781a64493ae095b6895b153c1', 'request_id': 'req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.943917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ea008781a64493ae095b6895b153c1', 'request_id': 'req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.960769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] The token cache is being invalidate because project f9ea008781a64493ae095b6895b153c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:40.961206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.961996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:40.963814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3766/7461] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:40 2022] DELETE /identity/v3/projects/f9ea008781a64493ae095b6895b153c1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:40.968220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:40.970016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:40.983642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9abac-f29f-441c-8a56-4f378af8026f tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:40.990828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:41.044676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.045304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.045586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.045880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.047330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8ada586d09934f36979aa3d2425aa45e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.048928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.072776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.073577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.100779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.101768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.104459 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3696/7462] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:16:40 2022] PATCH /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e => generated 278 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:41.108575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:41.120302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97dcd6-1c30-4fe5-ab00-2fbbe377efda tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:41.180264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.180828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.181112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.181386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/domains/8ada586d09934f36979aa3d2425aa45e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.192177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8ada586d09934f36979aa3d2425aa45e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.194010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.196269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:41.222697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.227260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8ada586d09934f36979aa3d2425aa45e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.273768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.280503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.290017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.305919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] The token cache is being invalidate because project 8ada586d09934f36979aa3d2425aa45e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:41.306283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.307029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.318056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.321738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.322596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8ada586d09934f36979aa3d2425aa45e', 'request_id': 'req-badb2227-b121-446c-b19d-b91e19feae75'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.328807 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3767/7463] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:16:41 2022] DELETE /identity/v3/domains/8ada586d09934f36979aa3d2425aa45e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:41.332738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:41.334746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:41.350113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd9f8d-f02e-4150-8cc7-c1ad485ec026 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:41.421001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.421635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.421913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.422190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f7b76de8f7df4539a273afee68d69d88` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.424821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f7b76de8f7df4539a273afee68d69d88)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.425523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.470635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role f7b76de8f7df4539a273afee68d69d88 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:41.470635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.470635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.470635 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3697/7464] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:41 2022] DELETE /identity/v3/roles/f7b76de8f7df4539a273afee68d69d88 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:41.473494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:41.475972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:41.489188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb2227-b121-446c-b19d-b91e19feae75 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:41.567334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.567997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.568282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.568675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/f1478b1a842f4378bbfe2cd5d9c9f475` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.570193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f1478b1a842f4378bbfe2cd5d9c9f475)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.571918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.609029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role f1478b1a842f4378bbfe2cd5d9c9f475 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:41.609700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.610478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.612401 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3768/7465] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:41 2022] DELETE /identity/v3/roles/f1478b1a842f4378bbfe2cd5d9c9f475 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:41.616507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:41.618349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:41.631004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250d1d6-7870-4c55-85a3-ef97845a7c9c tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:41.665642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:DELETE=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:41.692961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35572882ba4b46ffbd704e88a1330b2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7224530f72004967936e777eab92894f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.693588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.693874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.694147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] PATH_INFO: `/v3/roles/35bc718f22774c1295e4adda89703c51` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.695581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorizing `identity:delete_role(role_id=35bc718f22774c1295e4adda89703c51)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.697181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.735400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invalidating the token cache because role 35bc718f22774c1295e4adda89703c51 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:41.735708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.736443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-829bf00b-4fe7-4df5-96d4-cde7150d9f00 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.738308 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3698/7466] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:41 2022] DELETE /identity/v3/roles/35bc718f22774c1295e4adda89703c51 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:41.742813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:41.744372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:41.759638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:41.814602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cd2e-525e-4122-91f1-72d191c13db5 tempest-RolesV3TestJSON-418212841 tempest-RolesV3TestJSON-418212841-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:41.876483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:41.877118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.877398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.877672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] PATH_INFO: `/v3/users/5bc0116356b54ae1bf8b2d1e6a210add` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.878995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc0116356b54ae1bf8b2d1e6a210add)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:41.880943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:41.928048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc0116356b54ae1bf8b2d1e6a210add'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.928926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc0116356b54ae1bf8b2d1e6a210add'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.936569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc0116356b54ae1bf8b2d1e6a210add'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.949170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc0116356b54ae1bf8b2d1e6a210add'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.949864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc0116356b54ae1bf8b2d1e6a210add'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:41.962399 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3769/7467] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:41 2022] DELETE /identity/v3/users/5bc0116356b54ae1bf8b2d1e6a210add => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:41.967051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:41.967515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:41.967795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:41.994830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:42.000457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:42.068325 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3699/7468] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:42.078719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:42.080252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:42.135693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:42.154519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019ee45-cb1a-423d-a8d1-82c41e584659 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:42.158378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:42.159163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:42.159506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:42.159845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:42.170066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:42.171586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:42.182136 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3770/7469] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:42.459958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:42.461387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:42.474218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115d641-7ed0-47a5-a202-51a2c5b61230 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:42.479098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:42.479781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:42.480064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:42.480419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] PATH_INFO: `/v3/projects/b99f66d25bbc41d681972633d25a2a86` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:42.481918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99f66d25bbc41d681972633d25a2a86)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:42.483981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:42.532633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99f66d25bbc41d681972633d25a2a86', 'request_id': 'req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.539211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99f66d25bbc41d681972633d25a2a86', 'request_id': 'req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.547608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99f66d25bbc41d681972633d25a2a86', 'request_id': 'req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.559619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] The token cache is being invalidate because project b99f66d25bbc41d681972633d25a2a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:42.559908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.560526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.562000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3700/7470] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:42 2022] DELETE /identity/v3/projects/b99f66d25bbc41d681972633d25a2a86 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:42.565996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:42.576488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6d341-06b9-4a71-8766-394a4bcb4896 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:42.638122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:42.638796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:42.639078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:42.639392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] PATH_INFO: `/v3/users/35572882ba4b46ffbd704e88a1330b2a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:42.640803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] RBAC: Authorizing `identity:delete_user(user_id=35572882ba4b46ffbd704e88a1330b2a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:42.642774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:42.690373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35572882ba4b46ffbd704e88a1330b2a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.691124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35572882ba4b46ffbd704e88a1330b2a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.698838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35572882ba4b46ffbd704e88a1330b2a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.711672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35572882ba4b46ffbd704e88a1330b2a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.712357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-97de1459-8ae6-4ad0-bde3-8e9c94e888af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35572882ba4b46ffbd704e88a1330b2a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:42.725170 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3771/7471] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:42 2022] DELETE /identity/v3/users/35572882ba4b46ffbd704e88a1330b2a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:42.999496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.000957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:43.018250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb0f66-d23a-4b51-b65c-f93ce7fab431 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.040774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:43.079750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.080301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.080602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.080890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] PATH_INFO: `/v3/projects/7224530f72004967936e777eab92894f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.081991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7224530f72004967936e777eab92894f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.084481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.133833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7224530f72004967936e777eab92894f', 'request_id': 'req-53b119bd-141a-415f-b769-907d53584a9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:43.140156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7224530f72004967936e777eab92894f', 'request_id': 'req-53b119bd-141a-415f-b769-907d53584a9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:43.150013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7224530f72004967936e777eab92894f', 'request_id': 'req-53b119bd-141a-415f-b769-907d53584a9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:43.164971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] The token cache is being invalidate because project 7224530f72004967936e777eab92894f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:43.165331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:43.165932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-53b119bd-141a-415f-b769-907d53584a9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:43.167499 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3701/7472] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:42 2022] DELETE /identity/v3/projects/7224530f72004967936e777eab92894f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:43.173346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.173828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.173931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.195673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:43.196898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:43.283713 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3772/7473] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:43.292795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.293085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.293362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.313630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:43.315090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:43.324317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3702/7474] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:43.328602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.334430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:43.340979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a2080-6a03-4de3-b85b-657b5624ebff None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.347081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.347845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.348237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.348606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.349320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:43.349693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.351529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.366535 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3773/7475] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:43.367969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.377210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed957f3b-89f7-4c5d-a2af-9523dcdbce38 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.382682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.383270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.383553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.383826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.384458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.386369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.414203 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3703/7476] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:43.418418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.428817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905bc13-1a5f-4524-a399-1d288d8558d6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.493315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.494079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.494358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.495059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.495378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.497416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.520102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3774/7477] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:43.525089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.537877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc43a9-266e-4721-9259-5a50318e3021 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.543056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.543972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.544855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.545136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.545793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.547422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.561987 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3704/7478] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:43.563326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.574734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720eafd-0f04-4c63-8d1b-83732730898c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.579509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.580107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.580385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.580674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.581312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.583659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593dac17-0bba-4340-8ce2-e2b857d1ffa7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.594707 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3775/7479] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:43.599653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.614981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb9cf5-b6d6-4088-906b-a5e9d7164471 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.619758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.620413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.620695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.620964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] PATH_INFO: `/v3/projects/914aa9601bb344e79fa15d0bb0f2e4cd/users/c5c3559a63ff4fab90416916c3d8ddc3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.642800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=914aa9601bb344e79fa15d0bb0f2e4cd, user_id=c5c3559a63ff4fab90416916c3d8ddc3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.645974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.660421 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3705/7480] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:43 2022] PUT /identity/v3/projects/914aa9601bb344e79fa15d0bb0f2e4cd/users/c5c3559a63ff4fab90416916c3d8ddc3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:43.671054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.671344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.671621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.701025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] MFA Rules not processed for user `c5c3559a63ff4fab90416916c3d8ddc3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:43.702266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:43.775061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3776/7481] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:43.776306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.791574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7b182-f4fe-4d5b-b8d7-40743bc08822 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.796151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.796865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.797157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.797437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.798848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:create_application_credential(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.800134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:43.875779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:43.944008 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3706/7482] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:16:43 2022] POST /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials => generated 921 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:43.948401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:43.957683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9055410-1edd-49f9-8f93-e68226fa6ac1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:43.989467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:43.991046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:43.991380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:43.991686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/be0cbabdd1e94f408a46c6b84b5c7804` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:43.993898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:delete_access_rule(user_id=c5c3559a63ff4fab90416916c3d8ddc3, access_rule_id=be0cbabdd1e94f408a46c6b84b5c7804)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:43.997963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application session.delete(ref) Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application yield resource Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application next(self.gen) Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 21:16:44.055785 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise self.value Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application session.commit() Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.session.flush() Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 21:16:44.057017 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application compat.raise_( Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._synchronize( Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise AssertionError( Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' Sep 24 21:16:44.058234 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:16:44.060174 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:16:44.061244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3777/7483] 10.222.0.52 () {58 vars in 1362 bytes} [Sat Sep 24 21:16:43 2022] DELETE /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/be0cbabdd1e94f408a46c6b84b5c7804 => generated 89 bytes in 111 msecs (HTTP/1.1 403) 5 headers in 183 bytes (1 switches on core 0) Sep 24 21:16:44.063745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.077170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f32362-df50-4e50-ac71-af5fe7e7d287 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.082756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.083768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.084066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.084349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials/6bcd8097f93a49318cac846d32857b86` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.100086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:delete_application_credential(user_id=c5c3559a63ff4fab90416916c3d8ddc3, application_credential_id=6bcd8097f93a49318cac846d32857b86)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.101669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.132299 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3707/7484] 10.222.0.52 () {58 vars in 1395 bytes} [Sat Sep 24 21:16:44 2022] DELETE /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials/6bcd8097f93a49318cac846d32857b86 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:44.136656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.150382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25a59-90b9-4cee-aabb-9f0c082d8eec tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.155991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.156752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.157265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.157774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.159367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:list_access_rules(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.163016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.170904 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3778/7485] 10.222.0.52 () {58 vars in 1260 bytes} [Sat Sep 24 21:16:44 2022] GET /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules => generated 369 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:44.173745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.185927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ebda2-af58-4801-9a94-99448b975701 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.190388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.191067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.191351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.191633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/be0cbabdd1e94f408a46c6b84b5c7804` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.193464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:delete_access_rule(user_id=c5c3559a63ff4fab90416916c3d8ddc3, access_rule_id=be0cbabdd1e94f408a46c6b84b5c7804)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.194588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.217353 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3708/7486] 10.222.0.52 () {58 vars in 1362 bytes} [Sat Sep 24 21:16:44 2022] DELETE /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/be0cbabdd1e94f408a46c6b84b5c7804 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:44.221740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.236169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab8d0-aa78-423e-86fe-70ba859ebe72 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.242693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.243753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.244187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.244568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.245925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:list_access_rules(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.247620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.253364 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3779/7487] 10.222.0.52 () {58 vars in 1260 bytes} [Sat Sep 24 21:16:44 2022] GET /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules => generated 158 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:44.259395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.270002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79ac45-7a76-4536-b91e-70e55edd3eb6 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.278275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.278911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.279197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.279481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.280161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:create_application_credential(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.281322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.374298 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3709/7488] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:16:44 2022] POST /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials => generated 920 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:44.378387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.389254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254f02-45b6-4c0e-8aab-c110fa58be9c tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.394047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.394719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.395022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.395339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.396806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:list_access_rules(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.398300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.421704 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3780/7489] 10.222.0.52 () {58 vars in 1260 bytes} [Sat Sep 24 21:16:44 2022] GET /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules => generated 369 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:44.427587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.437112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16622732-4601-49d7-ba93-cc02916c3db5 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.442013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.443450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.443770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.444101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.444742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:create_application_credential(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.445902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.513700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:INSERT=11,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:44.529335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3710/7490] 10.222.0.52 () {60 vars in 1315 bytes} [Sat Sep 24 21:16:44 2022] POST /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/application_credentials => generated 919 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:44.531399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.543425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc0eb-0b08-4cb6-96a7-909a53d169cb tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.547863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5c3559a63ff4fab90416916c3d8ddc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '914aa9601bb344e79fa15d0bb0f2e4cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.548544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.548831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.549133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/5adb4d0f80004950a7270f548e9494a0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.550273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorizing `identity:get_access_rule(user_id=c5c3559a63ff4fab90416916c3d8ddc3, access_rule_id=5adb4d0f80004950a7270f548e9494a0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.552769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e4dd2-2c85-4fe6-9673-1f15c21cee8a tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.564187 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3781/7491] 10.222.0.52 () {58 vars in 1359 bytes} [Sat Sep 24 21:16:44 2022] GET /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3/access_rules/5adb4d0f80004950a7270f548e9494a0 => generated 229 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:44.565771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.647779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42af38-5bb8-4ed2-a2c0-772e8c8a2270 tempest-AccessRulesV3Test-662945081 tempest-AccessRulesV3Test-662945081-project] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.653621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.654351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.654664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.654936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] PATH_INFO: `/v3/users/c5c3559a63ff4fab90416916c3d8ddc3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.656301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c3559a63ff4fab90416916c3d8ddc3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.658209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.715857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c3559a63ff4fab90416916c3d8ddc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:44.716536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c3559a63ff4fab90416916c3d8ddc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:44.725186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c3559a63ff4fab90416916c3d8ddc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:44.732687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c3559a63ff4fab90416916c3d8ddc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:44.733357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c3559a63ff4fab90416916c3d8ddc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:44.781283 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3711/7492] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:44 2022] DELETE /identity/v3/users/c5c3559a63ff4fab90416916c3d8ddc3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:44.786398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.786863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.787376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.809400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:44.810753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:44.816023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:44.873442 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3782/7493] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:44.883716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:44.885217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:44.965788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe7e60-83ba-47ba-8b84-f31fb68b5d0c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:44.970000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:44.970653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:44.970933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:44.971204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:44.978380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:44.983079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:44.985081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4fb04-5a79-4767-8ef5-053f119aab29 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:44.996736 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3712/7494] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:45.274484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.276224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:45.289135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ff799-e161-46ac-b2af-42a6810c30c6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.294029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.294800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.295130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.295404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] PATH_INFO: `/v3/projects/914aa9601bb344e79fa15d0bb0f2e4cd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.296579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] RBAC: Authorizing `identity:delete_project(project_id=914aa9601bb344e79fa15d0bb0f2e4cd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.298567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.345407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '914aa9601bb344e79fa15d0bb0f2e4cd', 'request_id': 'req-10e4521b-70ca-47fd-8de3-741538f0622a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:45.351501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '914aa9601bb344e79fa15d0bb0f2e4cd', 'request_id': 'req-10e4521b-70ca-47fd-8de3-741538f0622a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:45.357630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '914aa9601bb344e79fa15d0bb0f2e4cd', 'request_id': 'req-10e4521b-70ca-47fd-8de3-741538f0622a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:45.370100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] The token cache is being invalidate because project 914aa9601bb344e79fa15d0bb0f2e4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:45.370363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:45.371002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-10e4521b-70ca-47fd-8de3-741538f0622a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:45.372469 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3783/7495] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:45 2022] DELETE /identity/v3/projects/914aa9601bb344e79fa15d0bb0f2e4cd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:45.377274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.377741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.377845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.398367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:45.399593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:45.474818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3713/7496] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:45.476556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.476904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.477189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.506093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:45.507419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:45.516755 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3784/7497] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:45.521845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.531916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c553d-852f-4d67-9cf3-7b7ac4ddf82a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.537426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.538041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.538362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.538624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.540179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:45.540179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.541219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.557778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3714/7498] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:45.562291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.575932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146318b5-dbfc-447d-b2ad-520c2f38ab36 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.582561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.583564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.584042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.584725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.584989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.587063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.646990 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3785/7499] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:45 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:45.647628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.658402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da310d3-f7a8-4046-a56c-9b68d5c5996d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.718372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.719015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.719293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.719561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.720194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.722197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.748130 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3715/7500] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:45 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:45.752192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.763257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40244d37-f4e7-411e-b9b1-b509fd044dbc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.767973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.768575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.768854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.769124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.769749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.771565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.782138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3786/7501] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:45.787431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.797538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051c5d6-d403-4e9c-a5b0-288bf362545f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.804411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.805094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.805393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.805656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] PATH_INFO: `/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe/users/94eb698a513341a7bcb7124e12e12a83/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.821779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee0bc5b14104a6f86edb77ba16e3ebe, user_id=94eb698a513341a7bcb7124e12e12a83, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.824575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.839171 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3716/7502] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:45 2022] PUT /identity/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe/users/94eb698a513341a7bcb7124e12e12a83/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:45.843172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.854615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153ba4c-a541-4b29-b4da-3ac97ea03df7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.909061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:45.917759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.918863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.919496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.920106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.921080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.922918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1419071-0ff3-497f-99f8-c3d934c99c9f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.934673 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3787/7503] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:45.938808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:45.949243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa11c3-9280-42d5-abe2-54ce77fd2842 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:45.953960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:45.954637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.954918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.955191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] PATH_INFO: `/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe/users/94eb698a513341a7bcb7124e12e12a83/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:45.957106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee0bc5b14104a6f86edb77ba16e3ebe, user_id=94eb698a513341a7bcb7124e12e12a83, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:45.960311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:45.970469 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3717/7504] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:45 2022] PUT /identity/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe/users/94eb698a513341a7bcb7124e12e12a83/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:45.982385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:45.982709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:45.982985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.012361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] MFA Rules not processed for user `94eb698a513341a7bcb7124e12e12a83`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:46.013595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:46.089689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3788/7505] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:46.094333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.130536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:46.160728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75399600-631f-4ccf-8fcf-883fbda82513 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.165884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.166473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.166785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.167057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.167677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.169561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.196080 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3718/7506] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:46 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:46.200134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.211295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec8ab-51bf-4aaa-aba9-20fcdd55e691 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.276394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.277030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.277213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.277486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.278118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.280068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.300868 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3789/7507] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:46 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:46.304983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.315262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015fbfd-4fcc-453e-a4c4-2e592aa8267f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.321163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.321785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.322060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.322326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.323029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.324644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.336260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3719/7508] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:46.340474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.350819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba410d0-5616-4f31-8fc9-0ee6c4198eae admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.355590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.356168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.356452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.356726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.357415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.359045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb5c0d-2ab0-4d1e-bfdd-1aef681dcbd9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.369491 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3790/7509] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:46.373672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.384795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d958fe9-c4a1-41e2-bcbd-cbde185fdd65 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.389663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.390315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.390623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.390895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] PATH_INFO: `/v3/projects/11781facbe0d415cacf74df93f185b62/users/56bf736f4af242a0af942198d673091b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.407611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11781facbe0d415cacf74df93f185b62, user_id=56bf736f4af242a0af942198d673091b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.410287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.424390 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3720/7510] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:46 2022] PUT /identity/v3/projects/11781facbe0d415cacf74df93f185b62/users/56bf736f4af242a0af942198d673091b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:46.434346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.434671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.434957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.465990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] MFA Rules not processed for user `56bf736f4af242a0af942198d673091b`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:46.467263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:46.513781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:46.537113 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3791/7511] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:46.548943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:46.550762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:46.633718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbece4-6a01-4950-a391-22d139b35fd6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:46.638386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:46.639039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:46.639273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:46.639703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:46.715139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:46.716781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dfc39-0fcb-4410-850b-28eb548f0dbe service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:46.731182 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3721/7512] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:47.414950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:47.484183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc3012-fd25-481c-b8f9-d28738d2a0c7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:47.496075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:47.496814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:47.497098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:47.497367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] PATH_INFO: `/v3/users/94eb698a513341a7bcb7124e12e12a83` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:47.498591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94eb698a513341a7bcb7124e12e12a83)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:47.500589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:47.552353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94eb698a513341a7bcb7124e12e12a83'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:47.553229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94eb698a513341a7bcb7124e12e12a83'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:47.561164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94eb698a513341a7bcb7124e12e12a83'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:47.574089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94eb698a513341a7bcb7124e12e12a83'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:47.574735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94eb698a513341a7bcb7124e12e12a83'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:47.587660 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3792/7513] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:47 2022] DELETE /identity/v3/users/94eb698a513341a7bcb7124e12e12a83 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:47.592197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceef79-0473-4238-b52c-786129803034 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:47.592653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceef79-0473-4238-b52c-786129803034 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:47.592735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dceef79-0473-4238-b52c-786129803034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:47.619046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4dceef79-0473-4238-b52c-786129803034 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:47.620277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4dceef79-0473-4238-b52c-786129803034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:47.653620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:47.687043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3722/7514] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:47.696751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:47.698177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:47.835826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651f56-887f-4d7c-beff-b87f505f1071 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:47.839959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:47.840581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:47.840967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:47.841584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:47.857030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:47.858904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:47.877687 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3793/7515] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:48.180369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceef79-0473-4238-b52c-786129803034 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:48.181996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4dceef79-0473-4238-b52c-786129803034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:48.195272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dceef79-0473-4238-b52c-786129803034 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:48.199978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:48.200610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:48.200921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:48.201194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] PATH_INFO: `/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:48.202331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee0bc5b14104a6f86edb77ba16e3ebe)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:48.204278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:48.252981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee0bc5b14104a6f86edb77ba16e3ebe', 'request_id': 'req-f40085d4-b81f-4764-a1d2-9b74820658c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.259271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee0bc5b14104a6f86edb77ba16e3ebe', 'request_id': 'req-f40085d4-b81f-4764-a1d2-9b74820658c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.267573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee0bc5b14104a6f86edb77ba16e3ebe', 'request_id': 'req-f40085d4-b81f-4764-a1d2-9b74820658c6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.278834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] The token cache is being invalidate because project fee0bc5b14104a6f86edb77ba16e3ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:48.279124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.279756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.281231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3723/7516] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:48 2022] DELETE /identity/v3/projects/fee0bc5b14104a6f86edb77ba16e3ebe => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:48.285187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:48.295184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7165-64a4-4652-8e7b-45fec4494cff service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:48.298829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:48.373863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:48.375065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:48.375065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:48.375065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] PATH_INFO: `/v3/users/56bf736f4af242a0af942198d673091b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:48.376255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bf736f4af242a0af942198d673091b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:48.378218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:48.445322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bf736f4af242a0af942198d673091b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.446803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bf736f4af242a0af942198d673091b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.461490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bf736f4af242a0af942198d673091b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.482129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bf736f4af242a0af942198d673091b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.483494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3aa099aa-8f29-46e3-a740-a19945e8fe0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bf736f4af242a0af942198d673091b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.502330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3794/7517] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:48 2022] DELETE /identity/v3/users/56bf736f4af242a0af942198d673091b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:48.779303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:48.780783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:48.794980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40085d4-b81f-4764-a1d2-9b74820658c6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:48.852376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:48.852939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:48.853216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:48.853478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] PATH_INFO: `/v3/projects/11781facbe0d415cacf74df93f185b62` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:48.854656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] RBAC: Authorizing `identity:delete_project(project_id=11781facbe0d415cacf74df93f185b62)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:48.856602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:48.903473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11781facbe0d415cacf74df93f185b62', 'request_id': 'req-258387af-cb1e-4fca-9d66-ca5161cd421c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.909190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11781facbe0d415cacf74df93f185b62', 'request_id': 'req-258387af-cb1e-4fca-9d66-ca5161cd421c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.911953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:48.921267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11781facbe0d415cacf74df93f185b62', 'request_id': 'req-258387af-cb1e-4fca-9d66-ca5161cd421c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.933262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] The token cache is being invalidate because project 11781facbe0d415cacf74df93f185b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:48.933548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.934191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-258387af-cb1e-4fca-9d66-ca5161cd421c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:48.935686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3724/7518] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:48 2022] DELETE /identity/v3/projects/11781facbe0d415cacf74df93f185b62 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:48.941386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:48.941674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:48.941975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:48.962830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:48.964033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:49.027124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3795/7519] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:49.034982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.035340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.035628 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.056257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:49.057461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:49.065694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3725/7520] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:49.070009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.079892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d263a-e502-4571-a0f5-dfcb1a7dba59 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.085299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.085897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.086176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.086450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.087117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:49.087421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.089008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.100961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3796/7521] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:49.105480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.115038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ff99-00c1-4fc1-8d60-afb6b61bf0ad None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.119735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.120342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.120618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.120890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.121543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.123472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.157939 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3726/7522] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:49.159301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.170355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47d778-f411-4d2c-b65f-232b2beeb9be admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.179915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:49.234196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.234848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.235299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.235571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.236199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.238174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.266167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3797/7523] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:49.267726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.279993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3c0e6-e838-45a4-8b08-63016a209ee7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.284767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.285381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.285926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.286504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.287173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.288783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.302417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3727/7524] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:49.303709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.314413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b1a6a-de62-47b6-a496-df1742dd5ad0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.320116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.320731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.321011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.321279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.321918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.323573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22851aea-4e98-4dcc-b05c-d212099ab75d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.336524 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3798/7525] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:49.340673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.351314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d874c70-c235-4cb4-aa51-5baf605781b6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.356210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.356817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.357092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.357360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] PATH_INFO: `/v3/projects/bd28c681381943529c3c3d4d8267c213/users/918a2994627f4376bcad16f9f7748345/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.375259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd28c681381943529c3c3d4d8267c213, user_id=918a2994627f4376bcad16f9f7748345, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.379868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.399231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3728/7526] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:49 2022] PUT /identity/v3/projects/bd28c681381943529c3c3d4d8267c213/users/918a2994627f4376bcad16f9f7748345/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:49.410947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.411367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.411653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.453712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] MFA Rules not processed for user `918a2994627f4376bcad16f9f7748345`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:49.455061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:49.543756 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3799/7527] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:49.548029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.647995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586f4a6-b7ae-4c48-9f27-d7e2e72d174a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.654165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.655242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.656613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.657134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.658282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.662356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.704142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3729/7528] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:49.709101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.721368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbafac-4c24-4a94-ba65-1c39ccc81cc1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.811279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.812099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.812382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.812845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.813955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.816339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.872200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3800/7529] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:49 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:49.873705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.888848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714319-b1cb-4a68-b4c9-d015e4eee22b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.895770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.896497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.896905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.897189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.898026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.900182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.915427 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3730/7530] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:49.916642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.927302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76619f85-7bc9-4dcc-bd8e-90bdeb5c0f33 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.932167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.932779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.933090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.933402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:49.934044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:49.935674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d6e3-fd66-42c1-966f-6d8833041b72 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:49.941668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:49.948822 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3801/7531] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:49.952312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:49.964073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1de32-735f-4aa0-9d73-44c978063421 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:49.969827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:49.970840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:49.971268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:49.971736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] PATH_INFO: `/v3/projects/94f29dc458f643eeaee3a4ea766fa116/users/84f890a31ab244539f0b020724058503/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.012508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f29dc458f643eeaee3a4ea766fa116, user_id=84f890a31ab244539f0b020724058503, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:50.015536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:50.038395 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3731/7532] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:49 2022] PUT /identity/v3/projects/94f29dc458f643eeaee3a4ea766fa116/users/84f890a31ab244539f0b020724058503/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:50.049747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:50.049747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:50.049747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.102299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:50.104808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.131035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] MFA Rules not processed for user `84f890a31ab244539f0b020724058503`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:50.132569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.175520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:50.238654 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3802/7533] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:50.253048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91d849-317e-43d8-a2e6-d86bd38adad1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:50.259710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:50.260421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:50.260706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:50.261052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.262958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:50.264078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.264288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.282304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:50.283939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66282da-026b-4183-8e49-fb47f7a9a974 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:50.299996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3732/7534] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:50.387076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f1db9-41e4-4b74-a088-ce9a6e540acc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:50.391720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:50.392347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:50.392625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:50.393083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.451967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:50.453328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:50.463376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3803/7535] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:16:50.600459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:50.600802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:50.601060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.643187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:50.645713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.729698 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3733/7536] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:50.823251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:50.824914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:50.917169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:50.946803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bef65-d2ff-4c4f-b97a-a628e14eeb80 service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:50.952983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:50.953690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:50.954032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:50.954368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:50.965089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:50.966860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00334817-9b43-48d3-a6b8-1bca13739563 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:50.976075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3804/7537] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:16:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:16:51.205717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:51.207924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:51.235150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac0c1-57f6-40a8-a8c8-020194a0533b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:51.245565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:51.246198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:51.246480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:51.246784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:51.271435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:51.275081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:51.298718 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3734/7538] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:51.383575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:51.383908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:51.384184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:51.408233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:51.412240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:51.422085 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3805/7539] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:51.437532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:51.453926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed901f4-0478-40e8-9131-99bc9bf4fe26 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:51.461394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:51.462274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:51.462790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:51.463409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:51.479970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:51.482613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:51.494154 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3735/7540] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:16:51.530525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:51.532277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:51.736720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:51.740067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8935-b6e2-4658-af37-a8406e185214 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:51.748744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:51.749579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:51.749925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:51.750463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:51.753133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:51.774642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:51.777951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756286ef-e261-48da-a6be-63dcce2cd698 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:51.804476 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3806/7541] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:51.880922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d9cf-87bc-4be8-b11f-bb97a3d4642c service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:51.892562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:51.893366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:51.893793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:51.894170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] PATH_INFO: `/v3/users/918a2994627f4376bcad16f9f7748345` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:51.897269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=918a2994627f4376bcad16f9f7748345)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:51.899673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:51.944479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:51.962012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918a2994627f4376bcad16f9f7748345'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:51.962794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918a2994627f4376bcad16f9f7748345'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:51.971237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918a2994627f4376bcad16f9f7748345'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:51.980294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918a2994627f4376bcad16f9f7748345'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:51.981079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918a2994627f4376bcad16f9f7748345'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.006415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3736/7542] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:51 2022] DELETE /identity/v3/users/918a2994627f4376bcad16f9f7748345 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:52.010884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:52.011250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:52.011599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:52.033337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:52.034695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:52.112901 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3807/7543] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:52.131734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:52.133755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:52.202323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b4ccf-772c-4800-992f-084e2c8318e1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:52.211315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:52.211961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:52.212199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:52.212471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:52.233046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:52.235762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:52.249118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:52.250028 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3737/7544] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:52.251821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:52.280183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121225b2-a64d-4cc4-a53c-bba4eed2dcf6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:52.285280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:52.286129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:52.286811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:52.288213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] PATH_INFO: `/v3/users/b53b6cad57584720aa0ecbe097fe92c9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:52.294099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53b6cad57584720aa0ecbe097fe92c9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:52.296105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:52.413875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53b6cad57584720aa0ecbe097fe92c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.414604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53b6cad57584720aa0ecbe097fe92c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.428433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:52.438626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53b6cad57584720aa0ecbe097fe92c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.466643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53b6cad57584720aa0ecbe097fe92c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.467515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53b6cad57584720aa0ecbe097fe92c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:52.485517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3808/7545] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:52 2022] DELETE /identity/v3/users/b53b6cad57584720aa0ecbe097fe92c9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:52.771336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:52.775345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:52.806652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:52.808159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:52.812407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffca2b-556c-4064-8eb8-e579735eaaa0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:52.906137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9896f6-131a-4791-9aab-5642a5b9db5b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:52.913984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:52.915171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:52.915653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:52.916180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] PATH_INFO: `/v3/projects/3601e0eb4f654079810627bef64cb0da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:52.917643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3601e0eb4f654079810627bef64cb0da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:52.920984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:52.922549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:52.923200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:52.923547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:52.923828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] PATH_INFO: `/v3/projects/bd28c681381943529c3c3d4d8267c213` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:52.925016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd28c681381943529c3c3d4d8267c213)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:52.927170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:53.024905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3601e0eb4f654079810627bef64cb0da', 'request_id': 'req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.026855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd28c681381943529c3c3d4d8267c213', 'request_id': 'req-8dfb12c8-f384-48e9-8305-182be4584cf9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.030089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:53.031983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3601e0eb4f654079810627bef64cb0da', 'request_id': 'req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.036599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd28c681381943529c3c3d4d8267c213', 'request_id': 'req-8dfb12c8-f384-48e9-8305-182be4584cf9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.041414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3601e0eb4f654079810627bef64cb0da', 'request_id': 'req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.046857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd28c681381943529c3c3d4d8267c213', 'request_id': 'req-8dfb12c8-f384-48e9-8305-182be4584cf9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.063576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] The token cache is being invalidate because project 3601e0eb4f654079810627bef64cb0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:53.064433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.065274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.066315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] The token cache is being invalidate because project bd28c681381943529c3c3d4d8267c213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:53.067244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.068839 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3809/7546] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:52 2022] DELETE /identity/v3/projects/3601e0eb4f654079810627bef64cb0da => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:53.069037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.070591 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3738/7547] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:52 2022] DELETE /identity/v3/projects/bd28c681381943529c3c3d4d8267c213 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:53.072562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:53.074646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:53.074770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:53.076405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:53.187387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ca0f6-cbe2-4267-a04c-a2544a27462c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:53.194871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:53.195832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:53.196104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:53.196382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] PATH_INFO: `/v3/users/d872d0d2c09344e7b9edbf8b2065e0a4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:53.197735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d872d0d2c09344e7b9edbf8b2065e0a4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:53.200690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:53.202588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:53.211118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb12c8-f384-48e9-8305-182be4584cf9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:53.228850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:53.230552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:53.231022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:53.231691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] PATH_INFO: `/v3/users/84f890a31ab244539f0b020724058503` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:53.233180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f890a31ab244539f0b020724058503)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:53.235512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:53.294064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd872d0d2c09344e7b9edbf8b2065e0a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.295087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd872d0d2c09344e7b9edbf8b2065e0a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.304796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd872d0d2c09344e7b9edbf8b2065e0a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.322008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f890a31ab244539f0b020724058503'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.323534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f890a31ab244539f0b020724058503'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.326699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd872d0d2c09344e7b9edbf8b2065e0a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.328005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd872d0d2c09344e7b9edbf8b2065e0a4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.333475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f890a31ab244539f0b020724058503'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.345255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3810/7548] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:53 2022] DELETE /identity/v3/users/d872d0d2c09344e7b9edbf8b2065e0a4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:53.346080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f890a31ab244539f0b020724058503'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.346828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f890a31ab244539f0b020724058503'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.372000 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3739/7549] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:53 2022] DELETE /identity/v3/users/84f890a31ab244539f0b020724058503 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:53.456740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:53.458584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:53.572189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0a5c-0f01-4e8e-9e8a-80e318ab1209 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:53.669138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:53.669894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:53.670274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:53.670655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:53.672545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:53.687868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:53.689580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:53.704530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3811/7550] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:16:53.736900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:53.744873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:53.821742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8c12d-295c-48c0-a8b4-3126889bb3a3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:53.826467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:53.827124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:53.827403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:53.827676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] PATH_INFO: `/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:53.828799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1bf27d27e2490592d10c5f8c15d75f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:53.830738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:53.832285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:53.884477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1bf27d27e2490592d10c5f8c15d75f', 'request_id': 'req-4b0047d6-9aae-419c-9bec-4b3059ae2438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.894316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1bf27d27e2490592d10c5f8c15d75f', 'request_id': 'req-4b0047d6-9aae-419c-9bec-4b3059ae2438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.903328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1bf27d27e2490592d10c5f8c15d75f', 'request_id': 'req-4b0047d6-9aae-419c-9bec-4b3059ae2438'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.915350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] The token cache is being invalidate because project 3d1bf27d27e2490592d10c5f8c15d75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:53.915641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.916424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4b0047d6-9aae-419c-9bec-4b3059ae2438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:53.918125 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3740/7551] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:53 2022] DELETE /identity/v3/projects/3d1bf27d27e2490592d10c5f8c15d75f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:53.981975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:53.983531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:54.002709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087f384-1965-41b4-8518-e686f0c2cc1f service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.015666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:54.078046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.078747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.079032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.079304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] PATH_INFO: `/v3/projects/94f29dc458f643eeaee3a4ea766fa116` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.084568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f29dc458f643eeaee3a4ea766fa116)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.086649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.152490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f29dc458f643eeaee3a4ea766fa116', 'request_id': 'req-6c091bb4-2273-48ef-9e25-b304aa52244e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:54.159669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f29dc458f643eeaee3a4ea766fa116', 'request_id': 'req-6c091bb4-2273-48ef-9e25-b304aa52244e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:54.170283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f29dc458f643eeaee3a4ea766fa116', 'request_id': 'req-6c091bb4-2273-48ef-9e25-b304aa52244e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:54.199964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] The token cache is being invalidate because project 94f29dc458f643eeaee3a4ea766fa116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:54.200324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:54.201042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6c091bb4-2273-48ef-9e25-b304aa52244e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:54.207592 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3812/7552] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:53 2022] DELETE /identity/v3/projects/94f29dc458f643eeaee3a4ea766fa116 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:54.209593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.209883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.210154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.252973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:54.254362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:54.340249 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3741/7553] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:54.351285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.351757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.352204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.376883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:54.378139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:54.396511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3813/7554] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:54.398040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.412699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e10c09-b0dd-4e60-a7db-e1eb1c96c835 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.419180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.419535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.420005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.420461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.421973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:54.421973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.424140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.442565 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3742/7555] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:16:54.445023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.456609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9bd9f-2f96-4d91-80af-958ad98eacec None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.464880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.465403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.465689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.465956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.466636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.468848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.500648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3814/7556] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:54.505860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.521887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aaed4-4d55-400b-98b4-b0fdb3352b2e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.596630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.597665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.598181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.599114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.601058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.603829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.637830 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3743/7557] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:54.639171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.651648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b748d-c9f2-4565-bd13-3e8f94409fb0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.659829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.660631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.660836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.661115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.661819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.663706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.675490 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3815/7558] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:54.679897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.690859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882593e-d072-44c7-9da6-cf322e06f5a9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.696527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.697145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.697429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.697705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.698348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.700012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8b1a4-832e-4b94-9fe0-a7210a234e42 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.701639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:54.720074 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3744/7559] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:54.721547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.733289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0807ea8-14ae-4089-8caf-cd21b94708ac admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:54.739302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:54.741546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.742100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.742387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] PATH_INFO: `/v3/projects/0ff648adc0384ed08c61882b126c3f71/users/4eb83aeb57df406b85c4e84e1525ab76/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.760078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff648adc0384ed08c61882b126c3f71, user_id=4eb83aeb57df406b85c4e84e1525ab76, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:54.762075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:54.780224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3816/7560] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:54 2022] PUT /identity/v3/projects/0ff648adc0384ed08c61882b126c3f71/users/4eb83aeb57df406b85c4e84e1525ab76/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:54.793217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:54.793531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:54.793842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:54.828481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] MFA Rules not processed for user `4eb83aeb57df406b85c4e84e1525ab76`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:54.830657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:54.923025 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3745/7561] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:54.928357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:54.986315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:55.013528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50aa52-788c-4104-b9f5-c9b3684e671f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.018645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.019275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.019560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.020451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.021109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.023141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.053245 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3817/7562] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:54 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:55.057841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.070802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06573bb9-d565-4fab-9d36-bec273e1136f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.134639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.135166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.135447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.135724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.136360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.138299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.164876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3746/7563] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:55 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:55.166102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.176826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3004-1616-45b1-8e41-9c3e07f738df admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.184755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.185892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.186530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.187112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.188374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.190569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.211048 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3818/7564] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:55.215020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.227241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734d278-1a1a-4435-90ee-b0a87abc298c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.232120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.232735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.233018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.233295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.234213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.235904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1b680-6618-4d77-9a68-998e9ff74507 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.246206 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3747/7565] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:55.250389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.261607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85005f-cd6c-4ed8-b059-d8766ef75082 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.266534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.267432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.267709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.268000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] PATH_INFO: `/v3/projects/3cbefb6284ce44729fa808b28b03ab58/users/ee98dda9bac9477d81b4200ee008be13/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.286429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cbefb6284ce44729fa808b28b03ab58, user_id=ee98dda9bac9477d81b4200ee008be13, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.289111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.303650 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3819/7566] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:55 2022] PUT /identity/v3/projects/3cbefb6284ce44729fa808b28b03ab58/users/ee98dda9bac9477d81b4200ee008be13/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:55.314031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.314536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.314671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.346887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] MFA Rules not processed for user `ee98dda9bac9477d81b4200ee008be13`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:55.348975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:55.415003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:55.431610 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3748/7567] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:55.442244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.444134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:55.535709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd7ebe-646a-4748-8a4a-4dd1b697dd55 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.539876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.540699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.541135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.541561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.631594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.634038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.653976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3820/7568] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:55.822797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:55.837437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc431e14-9d7c-4c35-9059-70120c5b422c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:55.841734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:55.842342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:55.842667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:55.842940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:55.855559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:55.857456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13f38-b222-4bdf-876f-1b547551a973 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:55.869389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3749/7569] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:57.299906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:57.352997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:57.382131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cbdd4-2822-4b63-8ceb-21a5bbef03e6 service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:57.394896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:57.395940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:57.396226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:57.396678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] PATH_INFO: `/v3/users/4eb83aeb57df406b85c4e84e1525ab76` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:57.397934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb83aeb57df406b85c4e84e1525ab76)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:57.399980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:57.456328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb83aeb57df406b85c4e84e1525ab76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:57.457269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb83aeb57df406b85c4e84e1525ab76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:57.468094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb83aeb57df406b85c4e84e1525ab76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:57.491640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb83aeb57df406b85c4e84e1525ab76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:57.492396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb83aeb57df406b85c4e84e1525ab76'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:57.505864 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3821/7570] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:57 2022] DELETE /identity/v3/users/4eb83aeb57df406b85c4e84e1525ab76 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:57.510743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:57.511316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:57.512014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:57.533896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:57.535722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:57.620439 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3750/7571] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:57.645859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:57.647391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:57.710878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f52ddb-5302-4b66-b209-99b562b3db79 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:57.714698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:57.715322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:57.715601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:57.715875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:57.728384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:57.729985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:57.743896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3822/7572] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:57.932817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:57.932817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:57.946984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abee12-c996-4c4e-bdac-775b41708ef9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:57.951885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:57.952520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:57.952799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:57.953082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] PATH_INFO: `/v3/users/992bb56657f340919192b7624025e253` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:57.954228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992bb56657f340919192b7624025e253)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:57.956206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.009367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992bb56657f340919192b7624025e253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.010275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992bb56657f340919192b7624025e253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.023595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992bb56657f340919192b7624025e253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.032249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992bb56657f340919192b7624025e253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.033073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992bb56657f340919192b7624025e253'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.052772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3751/7573] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:57 2022] DELETE /identity/v3/users/992bb56657f340919192b7624025e253 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:58.088184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.089901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.106477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3fc47-3377-40e6-a712-8869e3ed6ee7 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.176306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.176956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.177234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.177501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] PATH_INFO: `/v3/projects/0ff648adc0384ed08c61882b126c3f71` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.178695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff648adc0384ed08c61882b126c3f71)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:58.180627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.187486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:58.241678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff648adc0384ed08c61882b126c3f71', 'request_id': 'req-39ef364c-2e73-4247-afca-ea54e73006cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.248549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff648adc0384ed08c61882b126c3f71', 'request_id': 'req-39ef364c-2e73-4247-afca-ea54e73006cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.253814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.256064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.259856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff648adc0384ed08c61882b126c3f71', 'request_id': 'req-39ef364c-2e73-4247-afca-ea54e73006cb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.278585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] The token cache is being invalidate because project 0ff648adc0384ed08c61882b126c3f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:58.279188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.280020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.283730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3823/7574] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/projects/0ff648adc0384ed08c61882b126c3f71 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:58.290866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.292316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.292792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460fd6d-9ef5-4dfd-a890-5dc3a03f79b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.363146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:58.375042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef364c-2e73-4247-afca-ea54e73006cb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.376092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.376754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.377030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.377300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] PATH_INFO: `/v3/projects/6ecb9284c1e149c7a05e6065bf8762a9` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.379343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecb9284c1e149c7a05e6065bf8762a9)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:58.380371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.380968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.381267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.381526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] PATH_INFO: `/v3/users/ee98dda9bac9477d81b4200ee008be13` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.381654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.382705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee98dda9bac9477d81b4200ee008be13)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:58.384670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.430770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecb9284c1e149c7a05e6065bf8762a9', 'request_id': 'req-8fd1a50f-2461-4500-b62d-95293e68bda1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.433996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee98dda9bac9477d81b4200ee008be13'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.434726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee98dda9bac9477d81b4200ee008be13'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.436386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecb9284c1e149c7a05e6065bf8762a9', 'request_id': 'req-8fd1a50f-2461-4500-b62d-95293e68bda1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.443038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee98dda9bac9477d81b4200ee008be13'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.445464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecb9284c1e149c7a05e6065bf8762a9', 'request_id': 'req-8fd1a50f-2461-4500-b62d-95293e68bda1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.455881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee98dda9bac9477d81b4200ee008be13'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.456556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee98dda9bac9477d81b4200ee008be13'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.457214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] The token cache is being invalidate because project 6ecb9284c1e149c7a05e6065bf8762a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:58.457848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.458484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.459992 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3752/7575] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/projects/6ecb9284c1e149c7a05e6065bf8762a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:58.463910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.465806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.469547 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3824/7576] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/users/ee98dda9bac9477d81b4200ee008be13 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:58.536767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a50f-2461-4500-b62d-95293e68bda1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.600400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.601094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.601474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.601720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] PATH_INFO: `/v3/users/fc712fcf0f814a59bb52e16685ab1c9b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.603038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc712fcf0f814a59bb52e16685ab1c9b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:58.605269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.653235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:58.666737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc712fcf0f814a59bb52e16685ab1c9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.667472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc712fcf0f814a59bb52e16685ab1c9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.675145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc712fcf0f814a59bb52e16685ab1c9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.682838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc712fcf0f814a59bb52e16685ab1c9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.683558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc712fcf0f814a59bb52e16685ab1c9b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.702544 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3753/7577] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/users/fc712fcf0f814a59bb52e16685ab1c9b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:58.800046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.801580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.817347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77268-e4de-4783-b05a-58f1eaa476ca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.828329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:58.881448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.882011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.882295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.882596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] PATH_INFO: `/v3/projects/3cbefb6284ce44729fa808b28b03ab58` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.883765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cbefb6284ce44729fa808b28b03ab58)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:58.885671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:58.919942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:58.921525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:58.936259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fdb8d-1b23-4574-8f87-d8f0bdda59b9 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:58.951753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cbefb6284ce44729fa808b28b03ab58', 'request_id': 'req-57a9461b-d22e-4424-b56d-9cda83dbc6f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.958965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cbefb6284ce44729fa808b28b03ab58', 'request_id': 'req-57a9461b-d22e-4424-b56d-9cda83dbc6f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.965324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cbefb6284ce44729fa808b28b03ab58', 'request_id': 'req-57a9461b-d22e-4424-b56d-9cda83dbc6f4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.979875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] The token cache is being invalidate because project 3cbefb6284ce44729fa808b28b03ab58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:58.980170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.980808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-57a9461b-d22e-4424-b56d-9cda83dbc6f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:58.982864 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3825/7578] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/projects/3cbefb6284ce44729fa808b28b03ab58 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:58.989654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d16bd-6cb7-4acf-954e-4732b2e490b5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.989941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d16bd-6cb7-4acf-954e-4732b2e490b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.990217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d16bd-6cb7-4acf-954e-4732b2e490b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.996005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:58.996637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:58.996918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:58.997191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] PATH_INFO: `/v3/projects/0c0033b625ca4995bff138c485673fd8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:58.998312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0033b625ca4995bff138c485673fd8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.000246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.010641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d93d16bd-6cb7-4acf-954e-4732b2e490b5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.011865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d93d16bd-6cb7-4acf-954e-4732b2e490b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.021276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3826/7579] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:59.030476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.030796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.031223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.050583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.051879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.052374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0033b625ca4995bff138c485673fd8', 'request_id': 'req-6846d726-7bac-4c25-ae69-b0d14c4834a9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:59.058271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0033b625ca4995bff138c485673fd8', 'request_id': 'req-6846d726-7bac-4c25-ae69-b0d14c4834a9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:59.066983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0033b625ca4995bff138c485673fd8', 'request_id': 'req-6846d726-7bac-4c25-ae69-b0d14c4834a9'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:59.079222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] The token cache is being invalidate because project 0c0033b625ca4995bff138c485673fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:16:59.079513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:59.080168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6846d726-7bac-4c25-ae69-b0d14c4834a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:16:59.081659 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3754/7580] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:16:58 2022] DELETE /identity/v3/projects/0c0033b625ca4995bff138c485673fd8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:16:59.087362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85a2dc-f007-407e-85ce-be6a079adb75 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.087765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85a2dc-f007-407e-85ce-be6a079adb75 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.088046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85a2dc-f007-407e-85ce-be6a079adb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.108709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-df85a2dc-f007-407e-85ce-be6a079adb75 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.110042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-df85a2dc-f007-407e-85ce-be6a079adb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.114859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3827/7581] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:59.118053 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3755/7582] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:59.119695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.125265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.125545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.125814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.130243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee36de6-6e40-4b61-b0ff-8a838d9774ae None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.134965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.135497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.135771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.136039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.136697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:59.136993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.138651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.146105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.147335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.153748 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3828/7583] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:59.154929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.159103 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3756/7584] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:59.163922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.165301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e915131-434d-4a44-8478-81aaf34e496e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.170116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.170739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.171019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.171399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.172020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.173443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ecd59-827a-4358-aa01-8f98d280391f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.173904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.178394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.179080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.179361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.179640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.180310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:16:59.180613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.182254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.194694 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3757/7585] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:16:59.198983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.205612 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3829/7586] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:59.206454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.210698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837b096-043b-44a4-907f-4311b4340750 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.211614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:59.220919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:59.272717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.272972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591e5de-837f-4d74-af62-f681e94a0578 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.273303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.273582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.273851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.274486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.276780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.277955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.278572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.278855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.279127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.279778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.281676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.304612 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3758/7587] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:59.305490 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3830/7588] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:16:59.308784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.309499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.318578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495cb0b-ae86-4195-9936-68062a7e3b89 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.371035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a0b03-a706-497e-974e-9163343f79aa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.376059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.376633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.376812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.376935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.377097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.377368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.377497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.377784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.378003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.378645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.379660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.380951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.390700 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3759/7589] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:59.395062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.404226 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3831/7590] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:59.408190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd6620-a5ec-4d30-a5f8-b5998c955ef6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.408326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.413941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.414705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.415519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.415519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.415856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.417467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.419820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43af21-5d04-413c-82f5-c73d15bebf76 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.425219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.426133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.426701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.427097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.427851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.429585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.432552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3760/7591] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:16:59.433385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.445553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3832/7592] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:59.446777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.447040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5311f4-4f27-4c6f-ae79-6326e4f29852 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.454639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.455315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.455613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.455867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] PATH_INFO: `/v3/projects/08d3535f48224a4ca2c20808b7347e2a/users/de9fc973bdc840128baf7436f3d22d93/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.458752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5a46d-3e56-468c-b649-dd5fa67b8be6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.464834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.465729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.466221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.466773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.467789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.469615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.476265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d3535f48224a4ca2c20808b7347e2a, user_id=de9fc973bdc840128baf7436f3d22d93, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.478898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8c834-cd11-4512-a1d2-5363d0554d0a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.485896 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3833/7593] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:59.490594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.512529 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3761/7594] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:59 2022] PUT /identity/v3/projects/08d3535f48224a4ca2c20808b7347e2a/users/de9fc973bdc840128baf7436f3d22d93/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:59.514766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.515048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.515321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.515918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8c3ad-fec3-4929-a41f-fcffba5d8645 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.547410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] MFA Rules not processed for user `de9fc973bdc840128baf7436f3d22d93`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.548654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.590165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.590879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.591161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.591436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] PATH_INFO: `/v3/projects/d163ca6e94a144c992482cd1a6667a7b/users/ab391a7ffb98406b94d0ba676e35e799/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.593677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:59.610694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:16:59.618896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d163ca6e94a144c992482cd1a6667a7b, user_id=ab391a7ffb98406b94d0ba676e35e799, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.621761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdda5c-2d60-47b4-b9c3-e5117bbddd70 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.632858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3762/7595] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:16:59.637322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.637809 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3834/7596] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:59 2022] PUT /identity/v3/projects/d163ca6e94a144c992482cd1a6667a7b/users/ab391a7ffb98406b94d0ba676e35e799/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:59.642918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.643269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.643676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.649819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9886f-f688-4bc2-ad85-d25b2a00f990 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.664181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:16:59.665451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.710460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.711234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.711504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.711768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.712379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.714305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.725404 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3835/7597] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:16:59.736147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.737661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:16:59.744686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3763/7598] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:59.748617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.800005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecefd4e-df4f-4f67-9e27-bdf15832e462 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.812272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efdf0-ce23-4bca-b8b5-40686f29b975 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.817292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.817896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.818168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.818438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.819090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.821010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.849643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3764/7599] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:16:59 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:16:59.850840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.853902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.855280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.855561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.855973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.861333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f80734-9589-4732-be6f-920fa15a59f8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.866247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.867038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.867635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.867885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.867988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.868200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.868832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.870427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.878196 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3836/7600] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:16:59.885292 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3765/7601] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:16:59.886408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.896189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2270c-7c39-429c-81ba-bb64997fac70 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:16:59.902078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:16:59.902890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:16:59.903252 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:16:59.903626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] PATH_INFO: `/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5/users/ae01a196050e4f6a9ee694d303c37817/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:16:59.919694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ed584eb4eb43c99b6dcac5ac46f1c5, user_id=ae01a196050e4f6a9ee694d303c37817, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:16:59.922351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:16:59.936889 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3837/7602] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:16:59 2022] PUT /identity/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5/users/ae01a196050e4f6a9ee694d303c37817/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:16:59.940929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:16:59.952512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40442eb5-2c20-4676-8c67-34302a5423b5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:00.018363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:00.032729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:00.033343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:00.033619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:00.033886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:00.034555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:00.036364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c688363-89ad-4856-a2a0-c3cf186b3c18 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:00.049972 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3766/7603] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:00.051586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:00.064737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cd8a2-174c-4d4e-adf3-050d42013b0e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:00.073632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:00.074966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:00.075345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:00.075982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] PATH_INFO: `/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5/users/ae01a196050e4f6a9ee694d303c37817/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:00.078356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ed584eb4eb43c99b6dcac5ac46f1c5, user_id=ae01a196050e4f6a9ee694d303c37817, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:00.082918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:00.094136 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3838/7604] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:00 2022] PUT /identity/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5/users/ae01a196050e4f6a9ee694d303c37817/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:00.105488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:00.105778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:00.106060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:00.139064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] MFA Rules not processed for user `ae01a196050e4f6a9ee694d303c37817`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:00.140331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:00.218531 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3767/7605] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:00.237782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:00.240553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:00.250028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:00.319793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbabd8-684a-45dc-98c7-fa70d04a97a0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:00.323596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:00.324205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:00.324482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:00.324755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:00.390544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:00.391960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:00.402717 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3839/7606] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:00.877747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:00.898390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf46d7-6eff-4987-8fdd-554282b2fa4e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:00.903206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:00.904259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:00.904689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:00.905684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:00.922407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:00.924396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:00.937771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3768/7607] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:04.103189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:04.119854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaad3-fc98-47aa-b513-9b20fb28e6fa service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:04.129960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:04.130602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:04.130884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:04.131154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:04.135953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:04.154446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:04.156126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:04.177441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3840/7608] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:04.530974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:04.546276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bef0c3-8495-4d28-868d-836f2d11999a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:04.551915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:04.561026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:04.561370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:04.561682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:04.565042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:04.581056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:04.582589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:04.596836 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3769/7609] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:04.803709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:04.813403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:04.966405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a099d-f321-4895-8f65-757fdcbe652d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:04.983498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:04.984022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:04.984303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:04.984587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:04.986485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:05.013338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:05.015606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbbb12-dbd0-4d5e-9480-204ede5be59f service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:05.057494 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3841/7610] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:05.326374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:05.343341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964d25d-63a0-4630-8dcb-74e936382ad0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:05.352073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:05.353507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:05.353976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:05.354349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:05.357477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:05.495563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:05.497232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:05.515381 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3770/7611] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:06.064026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.064513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.064703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.142828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] MFA Rules not processed for user `ab391a7ffb98406b94d0ba676e35e799`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:06.144242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:06.187997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:06.266923 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3842/7612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:06.272612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:06.375674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:06.415271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95d26-ef29-4d57-8d57-1a8db1e70918 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:06.423178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:06.423806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.424084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.424354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.424986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:06.427759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:06.455329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3771/7613] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:06 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:06.459321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:06.471227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136da8d-5388-46a7-92cd-e1d070c7039c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:06.477940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:06.478748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.479035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.479309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.479967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:06.481679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e742f-4af7-477b-91b4-00c3feda07ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:06.492919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3843/7614] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:06.497071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:06.508928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96585836-8d73-47a9-b7a7-550002e484d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:06.515691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:06.516258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.516545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.516821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] PATH_INFO: `/v3/system/users/468d7d7dbc574a8c92e00d890ee60da3/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.535240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=468d7d7dbc574a8c92e00d890ee60da3, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:06.543615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:06.554751 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3772/7615] 10.222.0.52 () {60 vars in 1378 bytes} [Sat Sep 24 21:17:06 2022] PUT /identity/v3/system/users/468d7d7dbc574a8c92e00d890ee60da3/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:06.567585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.567974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.568241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.609260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] MFA Rules not processed for user `468d7d7dbc574a8c92e00d890ee60da3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:06.611022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:06.727092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3844/7616] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:06 2022] POST /identity/v3/auth/tokens => generated 2414 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 21:17:06.727827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:06.904057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1de11-f6df-4cce-9960-a504f365f019 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:06.951938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468d7d7dbc574a8c92e00d890ee60da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:06.952085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:06.952328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:06.952610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:06.953278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:06.954810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:06.967069 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3773/7617] 10.222.0.52 () {58 vars in 1110 bytes} [Sat Sep 24 21:17:06 2022] GET /identity/v3/services => generated 2191 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:17:06.969240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:07.103145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:07.136490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747cd6e-7dda-4739-ad95-03f36c29d319 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:07.185009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468d7d7dbc574a8c92e00d890ee60da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:07.186723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:07.187310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:07.187737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] PATH_INFO: `/v3/limits` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:07.188653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:07.190268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post Sep 24 21:17:07.256977 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.NoLimitReference Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. Sep 24 21:17:07.258032 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:17:07.258806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3845/7618] 10.222.0.52 () {60 vars in 1126 bytes} [Sat Sep 24 21:17:06 2022] POST /identity/v3/limits => generated 124 bytes in 290 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:17:07.263300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:07.282096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:07.416753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885571f-576c-4969-b8cb-d7893b976f2b None tempest-ImageQuotaTest-581846820-system-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:07.478083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468d7d7dbc574a8c92e00d890ee60da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:07.478782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:07.479092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:07.479347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:07.479966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:07.481116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:07.490959 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3774/7619] 10.222.0.52 () {58 vars in 1137 bytes} [Sat Sep 24 21:17:07 2022] GET /identity/v3/registered_limits => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:07.497503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:07.667903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b859cb-2cfc-4ab0-83f7-02a3ab10eaff None tempest-ImageQuotaTest-581846820-system-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:07.734692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468d7d7dbc574a8c92e00d890ee60da3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:07.735583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:07.735927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:07.736197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:07.736822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:07.738267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f6ff-5aa9-43c8-849b-28690890474b None tempest-ImageQuotaTest-581846820-system-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:07.741150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:07.746902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3846/7620] 10.222.0.52 () {58 vars in 1137 bytes} [Sat Sep 24 21:17:07 2022] GET /identity/v3/registered_limits => generated 129 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:10.166799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:10.178912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f990-a4e6-4429-a749-efc5badaab34 None tempest-ImageQuotaTest-581846820-system-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:10.184018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:10.184655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:10.186155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:10.186431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] PATH_INFO: `/v3/users/de9fc973bdc840128baf7436f3d22d93` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:10.187609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9fc973bdc840128baf7436f3d22d93)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:10.189693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:10.251949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9fc973bdc840128baf7436f3d22d93'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.252987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9fc973bdc840128baf7436f3d22d93'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.262552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9fc973bdc840128baf7436f3d22d93'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.271634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9fc973bdc840128baf7436f3d22d93'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.272373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9fc973bdc840128baf7436f3d22d93'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.291569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:10.312522 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3775/7621] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:10 2022] DELETE /identity/v3/users/de9fc973bdc840128baf7436f3d22d93 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:10.317576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:10.318092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:10.318243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:10.337488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:10.338744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:10.421217 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3847/7622] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:17:10.428651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:10.430192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:10.497761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e14cb-ec00-4f1d-9877-07a09b5006b6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:10.503681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:10.506032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:10.508183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:10.508469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:10.520141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:10.522646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:10.540219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3776/7623] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:10.743403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:10.744982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:10.761179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b37f21-8f82-4959-b3ab-210dba20aed9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:10.766711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:10.767362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:10.767698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:10.767969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] PATH_INFO: `/v3/projects/08d3535f48224a4ca2c20808b7347e2a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:10.770773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d3535f48224a4ca2c20808b7347e2a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:10.773326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:10.838279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d3535f48224a4ca2c20808b7347e2a', 'request_id': 'req-b63ab96b-b410-46f7-b889-da2679c3c243'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.844543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d3535f48224a4ca2c20808b7347e2a', 'request_id': 'req-b63ab96b-b410-46f7-b889-da2679c3c243'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.852785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d3535f48224a4ca2c20808b7347e2a', 'request_id': 'req-b63ab96b-b410-46f7-b889-da2679c3c243'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.870550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] The token cache is being invalidate because project 08d3535f48224a4ca2c20808b7347e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:10.871279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.872282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:10.874015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3848/7624] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:10 2022] DELETE /identity/v3/projects/08d3535f48224a4ca2c20808b7347e2a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:10.878110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:10.888794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8c310-9669-4373-8b4d-c563931155a8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:10.954834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:10.955624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:10.956003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:10.957138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] PATH_INFO: `/v3/users/ae01a196050e4f6a9ee694d303c37817` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:10.958473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae01a196050e4f6a9ee694d303c37817)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:10.960686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:11.020841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae01a196050e4f6a9ee694d303c37817'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.021846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae01a196050e4f6a9ee694d303c37817'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.031642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae01a196050e4f6a9ee694d303c37817'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.041503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae01a196050e4f6a9ee694d303c37817'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.042276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d5fe5ad6-a606-48e5-a055-582b3f57172a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae01a196050e4f6a9ee694d303c37817'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.061717 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3777/7625] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:10 2022] DELETE /identity/v3/users/ae01a196050e4f6a9ee694d303c37817 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:11.275841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:11.277645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:11.317429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ab96b-b410-46f7-b889-da2679c3c243 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:11.445838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:11.446746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.446948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.447355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] PATH_INFO: `/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.448674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ed584eb4eb43c99b6dcac5ac46f1c5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:11.450828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:11.480506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:11.514566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ed584eb4eb43c99b6dcac5ac46f1c5', 'request_id': 'req-689057dd-d6f4-429e-909c-bdecab16ab11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.522205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ed584eb4eb43c99b6dcac5ac46f1c5', 'request_id': 'req-689057dd-d6f4-429e-909c-bdecab16ab11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.538048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ed584eb4eb43c99b6dcac5ac46f1c5', 'request_id': 'req-689057dd-d6f4-429e-909c-bdecab16ab11'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.553575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] The token cache is being invalidate because project 59ed584eb4eb43c99b6dcac5ac46f1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:11.554073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.554820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-689057dd-d6f4-429e-909c-bdecab16ab11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:11.556643 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3849/7626] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:11 2022] DELETE /identity/v3/projects/59ed584eb4eb43c99b6dcac5ac46f1c5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:11.561630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.562150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.562150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.590701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:11.593095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:11.618166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:11.713041 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3778/7627] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:11.715932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.716409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.716547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.741666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:11.743240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:11.756565 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3850/7628] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:11.756923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:11.767342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a7aac-5ce9-47ba-9817-67c077251b8e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:11.773787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:11.774402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.774715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.774986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.776026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:17:11.776333 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:11.777941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:11.795584 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3779/7629] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:17:11.799935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:11.810139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c685a6-075e-4005-bfd8-7807a3241d70 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:11.815416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:11.817004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.817451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.817720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.818730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:11.820354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:11.870104 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3851/7630] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:11 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:11.874193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:11.900956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4ba78-b4d9-481d-a420-b8178070000a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:11.974304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:11.974998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:11.975329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:11.975601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:11.976229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:11.978522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.002006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3780/7631] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:11 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:12.006899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.017428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fd4-f1b5-448d-9ba9-34b77bb9eb3b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.023640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.024576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.025033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.025308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.026236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.027944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.041665 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3852/7632] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:17:12.043059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.054291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953ab9a-3106-4bae-87a7-cb4d82d121c3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.060447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.061065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.061342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.061612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.062265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.063994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636fa74-b57f-462e-986a-0110095cf9d8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.075776 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3781/7633] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:12.080077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.091476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b6711-1da2-493c-9a17-19a10945253e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.096269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.097045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.097427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.097711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] PATH_INFO: `/v3/projects/f1b687c5b51f4342be3df78e4fdeb9db/users/4c2ea1e3532d4cbea25ad88267230076/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.112457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b687c5b51f4342be3df78e4fdeb9db, user_id=4c2ea1e3532d4cbea25ad88267230076, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.115782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.128955 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3853/7634] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:12 2022] PUT /identity/v3/projects/f1b687c5b51f4342be3df78e4fdeb9db/users/4c2ea1e3532d4cbea25ad88267230076/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:12.139879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.140187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.140461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.170160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] MFA Rules not processed for user `4c2ea1e3532d4cbea25ad88267230076`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:12.171429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.292365 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3782/7635] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:12.295989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.378036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a007969-3f3c-4653-a48e-488f63da1aa9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.384441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.385141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.385422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.385694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.386338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.388317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.418694 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3854/7636] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:12 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:12.422464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.440207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a3ddb-c470-4955-a082-a05a2567ef84 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.450480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:12.450863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.452599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.502989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:12.514357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.517283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.517283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.517283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.517283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.518331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.543999 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3783/7637] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:12 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:12.544175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e40cc-0e45-40e3-ad29-41b3624a6f1b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.545041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.546199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.546871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.547158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.547554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] PATH_INFO: `/v3/users/ab391a7ffb98406b94d0ba676e35e799` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.548695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab391a7ffb98406b94d0ba676e35e799)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.550764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.554960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e81e4-8ad6-45c2-95b8-0971ee9b73c0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.559673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.560260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.560537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.560807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.561438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.563072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.574846 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3784/7638] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:17:12.578737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.593275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9f0ee-9096-45f9-94fd-70bf79ad1a44 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.602571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.603826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.604178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.604504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] PATH_INFO: `/v3/projects/8a9bd7a739294038bd1a4322aeb1e122/users/20ff4e364a434f379c3efb29dffccf9d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.622452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab391a7ffb98406b94d0ba676e35e799'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:12.623369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab391a7ffb98406b94d0ba676e35e799'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:12.624064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9bd7a739294038bd1a4322aeb1e122, user_id=20ff4e364a434f379c3efb29dffccf9d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.627122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.635792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab391a7ffb98406b94d0ba676e35e799'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:12.651823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab391a7ffb98406b94d0ba676e35e799'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:12.652172 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3785/7639] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:12 2022] PUT /identity/v3/projects/8a9bd7a739294038bd1a4322aeb1e122/users/20ff4e364a434f379c3efb29dffccf9d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:12.652704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab391a7ffb98406b94d0ba676e35e799'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:12.655924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.657636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.665523 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3855/7640] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:12 2022] DELETE /identity/v3/users/ab391a7ffb98406b94d0ba676e35e799 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:12.666392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.669944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.744577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406d49-24cf-4ca3-8b04-e6d48e59205d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.756033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb504a1-e334-4198-a016-cf41d0178a5e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.761331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.761946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.762235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.762518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.763250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.766809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.778275 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3856/7641] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:12.782577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:12.795161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722e7f9-06e9-4d89-b21c-5e0a33ede84d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:12.801549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.802198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.802468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.802770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.803654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:12.804319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.804598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.804870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] PATH_INFO: `/v3/projects/8a9bd7a739294038bd1a4322aeb1e122/users/20ff4e364a434f379c3efb29dffccf9d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.805812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.806953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9bd7a739294038bd1a4322aeb1e122, user_id=20ff4e364a434f379c3efb29dffccf9d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.809635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e64dd-51e4-4af9-9a1c-1abf0256bf28 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.822031 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3857/7642] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:12 2022] PUT /identity/v3/projects/8a9bd7a739294038bd1a4322aeb1e122/users/20ff4e364a434f379c3efb29dffccf9d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:12.854523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:12.886418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:12.888218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:12.903756 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3786/7643] 10.222.0.52 () {60 vars in 1396 bytes} [Sat Sep 24 21:17:12 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3274 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:12.922866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:12.923158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:12.923429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:12.953882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] MFA Rules not processed for user `20ff4e364a434f379c3efb29dffccf9d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:12.955908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:12.995298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:13.011587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:13.013265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:13.041907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422be885-2367-49ab-b636-de73a3ee08e4 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:13.051947 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3858/7644] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:13.062155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:13.103093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:13.103780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:13.104053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:13.104331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] PATH_INFO: `/v3/projects/d163ca6e94a144c992482cd1a6667a7b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:13.105465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d163ca6e94a144c992482cd1a6667a7b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:13.107453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:13.120851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28619aa0-ff1b-4f10-ae0e-47800cbeb431 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:13.125113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:13.125692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:13.126028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:13.126256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:13.136402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:13.137848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:13.148498 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3859/7645] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:17:13.161143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd163ca6e94a144c992482cd1a6667a7b', 'request_id': 'req-34a5ebc0-242d-4bda-b259-dd8c270ad431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.167433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd163ca6e94a144c992482cd1a6667a7b', 'request_id': 'req-34a5ebc0-242d-4bda-b259-dd8c270ad431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.176075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd163ca6e94a144c992482cd1a6667a7b', 'request_id': 'req-34a5ebc0-242d-4bda-b259-dd8c270ad431'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.187836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] The token cache is being invalidate because project d163ca6e94a144c992482cd1a6667a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:13.188150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.188788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.190270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3787/7646] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:13 2022] DELETE /identity/v3/projects/d163ca6e94a144c992482cd1a6667a7b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:13.195108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:13.207939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265c18-59df-4864-8937-863a6029e881 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:13.266203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:13.267133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:13.267415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:13.267799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] PATH_INFO: `/v3/users/468d7d7dbc574a8c92e00d890ee60da3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:13.269002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468d7d7dbc574a8c92e00d890ee60da3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:13.270975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:13.329356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468d7d7dbc574a8c92e00d890ee60da3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.329923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468d7d7dbc574a8c92e00d890ee60da3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.338770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468d7d7dbc574a8c92e00d890ee60da3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.339942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:13.354357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468d7d7dbc574a8c92e00d890ee60da3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.355000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9faaed9f-bc06-41cc-aecc-4e1571a0b8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468d7d7dbc574a8c92e00d890ee60da3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:13.368764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3860/7647] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:13 2022] DELETE /identity/v3/users/468d7d7dbc574a8c92e00d890ee60da3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:14.991694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:14.993118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:15.007989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5ebc0-242d-4bda-b259-dd8c270ad431 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:15.062169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:15.062826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:15.063112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:15.063382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] PATH_INFO: `/v3/users/4c2ea1e3532d4cbea25ad88267230076` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:15.064493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2ea1e3532d4cbea25ad88267230076)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:15.066667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:15.076573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:15.115865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2ea1e3532d4cbea25ad88267230076'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.116518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2ea1e3532d4cbea25ad88267230076'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.123988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2ea1e3532d4cbea25ad88267230076'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.131318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2ea1e3532d4cbea25ad88267230076'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.131989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2ea1e3532d4cbea25ad88267230076'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.149589 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3788/7648] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:14 2022] DELETE /identity/v3/users/4c2ea1e3532d4cbea25ad88267230076 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:15.154261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:15.154714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:15.154857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:15.174455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:15.176047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:15.243680 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3861/7649] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:17:15.253997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:15.255539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:15.318487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b27a-2927-437d-a1df-2870ca5f61cd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:15.322415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:15.323124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:15.323476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:15.323815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:15.333951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:15.335516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:15.345670 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3789/7650] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:15.625917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:15.627495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:15.641292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041aed8-5a46-45b6-855d-5d749def3b2c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:15.649931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:15.650911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:15.651228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:15.651510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] PATH_INFO: `/v3/projects/f1b687c5b51f4342be3df78e4fdeb9db` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:15.652648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b687c5b51f4342be3df78e4fdeb9db)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:15.654639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:15.701878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b687c5b51f4342be3df78e4fdeb9db', 'request_id': 'req-92cb202f-2179-4a21-b215-b4ea020cabce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.708449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b687c5b51f4342be3df78e4fdeb9db', 'request_id': 'req-92cb202f-2179-4a21-b215-b4ea020cabce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.719345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b687c5b51f4342be3df78e4fdeb9db', 'request_id': 'req-92cb202f-2179-4a21-b215-b4ea020cabce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.732773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] The token cache is being invalidate because project f1b687c5b51f4342be3df78e4fdeb9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:15.733062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.733678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.735214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3862/7651] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:15 2022] DELETE /identity/v3/projects/f1b687c5b51f4342be3df78e4fdeb9db => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:15.740108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:15.749770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1b84-ea73-443f-83ed-9bb1759e2f12 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:15.805820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:15.806450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:15.806761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:15.807035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] PATH_INFO: `/v3/users/20ff4e364a434f379c3efb29dffccf9d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:15.808152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ff4e364a434f379c3efb29dffccf9d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:15.810109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:15.841883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:15.855844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ff4e364a434f379c3efb29dffccf9d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.856487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ff4e364a434f379c3efb29dffccf9d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.863867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ff4e364a434f379c3efb29dffccf9d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.870836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ff4e364a434f379c3efb29dffccf9d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.871466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-26cd1063-ea8e-45e7-beff-20de0dac6231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ff4e364a434f379c3efb29dffccf9d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:15.888521 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3790/7652] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:15 2022] DELETE /identity/v3/users/20ff4e364a434f379c3efb29dffccf9d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:16.163275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.164758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:16.178198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb202f-2179-4a21-b215-b4ea020cabce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.229216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:16.237437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.238530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.238967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.239581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] PATH_INFO: `/v3/projects/8a9bd7a739294038bd1a4322aeb1e122` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.240661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9bd7a739294038bd1a4322aeb1e122)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.243060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.294288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9bd7a739294038bd1a4322aeb1e122', 'request_id': 'req-791305a6-ba29-4d10-8f85-616e69303c3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:16.299850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9bd7a739294038bd1a4322aeb1e122', 'request_id': 'req-791305a6-ba29-4d10-8f85-616e69303c3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:16.306333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9bd7a739294038bd1a4322aeb1e122', 'request_id': 'req-791305a6-ba29-4d10-8f85-616e69303c3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:16.320408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] The token cache is being invalidate because project 8a9bd7a739294038bd1a4322aeb1e122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:16.320700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:16.321344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-791305a6-ba29-4d10-8f85-616e69303c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:16.323165 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3863/7653] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:16 2022] DELETE /identity/v3/projects/8a9bd7a739294038bd1a4322aeb1e122 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:16.328288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.328814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.328814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.353897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:16.355256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:16.417075 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3791/7654] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:17:16.424843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.425296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.425688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.447510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:16.448723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:16.457517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3864/7655] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:16.461700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.471370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b6c94-769d-4671-8bc1-c7e05680667c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.476014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.476611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.476884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.477151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.477782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:17:16.478077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.479716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.492062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3792/7656] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:16.497435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.507893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104e797-1bad-47fd-af7a-63b4de585462 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.512641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.513276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.513564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.513837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.514523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.516472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.545745 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3865/7657] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:17:16.549916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.560369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af31039-ccc8-4352-93e1-0691e6c5de9f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.626543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.627140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.627416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.627686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.634813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.634813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.653517 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3793/7658] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/users => generated 277 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:16.657347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.668750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3566195-cc93-4a73-903b-d8bf06d87f1c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.673590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.674177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.674453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.674772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.676237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.677802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.689284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3866/7659] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:16.694484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.705131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ed41-6edd-4c03-bcac-5984502a5e12 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.711799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.712436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.712710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.712979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.713616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.715294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264e959-e3a4-41bf-8e3c-00100cddb250 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.726898 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3794/7660] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:16.731108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.741663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622d9a1-4212-40c3-a625-a700cdbeb413 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.746243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.746918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.747195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.747466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] PATH_INFO: `/v3/projects/1e54513054814ab18c140877b63ec20e/users/0459f0c985ca45609e50b51c168b598e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.762575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e54513054814ab18c140877b63ec20e, user_id=0459f0c985ca45609e50b51c168b598e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.765209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.778725 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3867/7661] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:16 2022] PUT /identity/v3/projects/1e54513054814ab18c140877b63ec20e/users/0459f0c985ca45609e50b51c168b598e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:16.789093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.789379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.789653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.819935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] MFA Rules not processed for user `0459f0c985ca45609e50b51c168b598e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:16.821058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:16.824995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:16.888566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3795/7662] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:16.893298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:16.954564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b23483-0023-4952-8535-25e60f68fd7e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:16.959343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:16.959947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:16.960227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:16.960500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:16.961130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:16.963106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:16.989715 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3868/7663] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/projects => generated 357 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:17:16.993763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.004698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5f1a-d0ee-4baa-850e-651fff326fe0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.061173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.061771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.062045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.062314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.062972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.064876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.086023 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3796/7664] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:16 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:17.089948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.101108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3739eb-4684-4f0e-a6cf-5c75ec043e30 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.105740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.106327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.106641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.106909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.107536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.109112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.110336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:17.122281 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3869/7665] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:17:17.126276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.136247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c820a3-b3fc-47fa-a081-3638373c396e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.140876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.141510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.141789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.142057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] PATH_INFO: `/v3/projects/9d18b8cbc0fb49b793cada5416f646ea/users/235c24cb205c47cfaf9befb19095ebc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.157147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d18b8cbc0fb49b793cada5416f646ea, user_id=235c24cb205c47cfaf9befb19095ebc9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.160355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.172969 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3797/7666] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:17 2022] PUT /identity/v3/projects/9d18b8cbc0fb49b793cada5416f646ea/users/235c24cb205c47cfaf9befb19095ebc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:17.177367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.187397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7af665-46d5-4b8c-8442-cad3da92ef72 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.245431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.246028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.246327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.246632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.247257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.248894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a0174-c95b-4263-bdf6-387ddbc24aaa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.259030 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3870/7667] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:17.263164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.274231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c42ab-cfe1-4151-b3b9-19365a567e02 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.279069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.279735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.280012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.280285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] PATH_INFO: `/v3/projects/9d18b8cbc0fb49b793cada5416f646ea/users/235c24cb205c47cfaf9befb19095ebc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.282180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d18b8cbc0fb49b793cada5416f646ea, user_id=235c24cb205c47cfaf9befb19095ebc9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.284907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.295397 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3798/7668] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:17 2022] PUT /identity/v3/projects/9d18b8cbc0fb49b793cada5416f646ea/users/235c24cb205c47cfaf9befb19095ebc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:17.306871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993ca3-d220-48a7-88af-333986881b1a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.307154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993ca3-d220-48a7-88af-333986881b1a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.307424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993ca3-d220-48a7-88af-333986881b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.336761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-da993ca3-d220-48a7-88af-333986881b1a None None] MFA Rules not processed for user `235c24cb205c47cfaf9befb19095ebc9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:17.337921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-da993ca3-d220-48a7-88af-333986881b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:17.406195 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3871/7669] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:17.411417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.422108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06babba9-5be6-4344-be36-001d0e734f74 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.427988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '235c24cb205c47cfaf9befb19095ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d18b8cbc0fb49b793cada5416f646ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.428606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.428914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.429185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.429825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:17:17.430245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.431922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.445321 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3799/7670] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:17.449447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da993ca3-d220-48a7-88af-333986881b1a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.458821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da993ca3-d220-48a7-88af-333986881b1a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.464234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '235c24cb205c47cfaf9befb19095ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d18b8cbc0fb49b793cada5416f646ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.464835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.465116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.465506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.466137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.468077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.497083 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3872/7671] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:17 2022] POST /identity/v3/projects => generated 349 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:17.501271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.512528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58decd-4538-4cc8-841b-1195f31ffb96 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.553911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:17.570639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '235c24cb205c47cfaf9befb19095ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d18b8cbc0fb49b793cada5416f646ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.571243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.571523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.571812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.572435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:17:17.572738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.574338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.585867 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3800/7672] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:17.595897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.598199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:17.662916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b82052-de8b-431d-b2b5-d9ca17abda62 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.666793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.667410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.667687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.668021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.679459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.680931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.685111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:17.693188 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3873/7673] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:17.854161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:17.864982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d40189-7064-4ca9-8c41-e531169d7436 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:17.869608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '235c24cb205c47cfaf9befb19095ebc9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9d18b8cbc0fb49b793cada5416f646ea', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:17.870229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:17.870539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:17.870823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] PATH_INFO: `/v3/projects/86ea086c8f1e47cba5dd0e26e2fac6db` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:17.871928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorizing `identity:delete_project(project_id=86ea086c8f1e47cba5dd0e26e2fac6db)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:17.873884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:17.919553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ea086c8f1e47cba5dd0e26e2fac6db', 'request_id': 'req-e86a1e77-d865-4c28-ba04-bf34bb073064'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:17.925667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ea086c8f1e47cba5dd0e26e2fac6db', 'request_id': 'req-e86a1e77-d865-4c28-ba04-bf34bb073064'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:17.933051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ea086c8f1e47cba5dd0e26e2fac6db', 'request_id': 'req-e86a1e77-d865-4c28-ba04-bf34bb073064'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:17.945050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] The token cache is being invalidate because project 86ea086c8f1e47cba5dd0e26e2fac6db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:17.945336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:17.945961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e86a1e77-d865-4c28-ba04-bf34bb073064 tempest-QuotasTest-1525479882 tempest-QuotasTest-1525479882-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:17.947445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3801/7674] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:17 2022] DELETE /identity/v3/projects/86ea086c8f1e47cba5dd0e26e2fac6db => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:18.060031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:18.122720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371780f-dab8-43aa-a3fc-ef151117bcce service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:18.188392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:18.189035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:18.189323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:18.189599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] PATH_INFO: `/v3/users/0459f0c985ca45609e50b51c168b598e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:18.190908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0459f0c985ca45609e50b51c168b598e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:18.192814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:18.242765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0459f0c985ca45609e50b51c168b598e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.243419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0459f0c985ca45609e50b51c168b598e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.250518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0459f0c985ca45609e50b51c168b598e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.262428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0459f0c985ca45609e50b51c168b598e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.263115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0459f0c985ca45609e50b51c168b598e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.276146 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3874/7675] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:18 2022] DELETE /identity/v3/users/0459f0c985ca45609e50b51c168b598e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:18.280606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:18.280889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:18.281162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:18.301650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:18.302873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:18.362389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3802/7676] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:18.372363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:18.373841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:18.403181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:18.436459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558777a-67fb-4d8f-b538-bf830a29023b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:18.440153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:18.441050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:18.441400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:18.441672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:18.450660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:18.452082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:18.462264 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3875/7677] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:17:18.740367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:18.741718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:18.754301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d911-7cda-47e9-92c4-13cba48f5481 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:18.759362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:18.760141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:18.760351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:18.760629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] PATH_INFO: `/v3/projects/1e54513054814ab18c140877b63ec20e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:18.762490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e54513054814ab18c140877b63ec20e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:18.764559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:18.813996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e54513054814ab18c140877b63ec20e', 'request_id': 'req-5e87c441-8205-4dd9-8c51-1f51c6f8435d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.818929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:18.820503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e54513054814ab18c140877b63ec20e', 'request_id': 'req-5e87c441-8205-4dd9-8c51-1f51c6f8435d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.836616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e54513054814ab18c140877b63ec20e', 'request_id': 'req-5e87c441-8205-4dd9-8c51-1f51c6f8435d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.849212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] The token cache is being invalidate because project 1e54513054814ab18c140877b63ec20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:18.849505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.850160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.851706 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3803/7678] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:18 2022] DELETE /identity/v3/projects/1e54513054814ab18c140877b63ec20e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:18.855630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:18.866479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301e7a8-7e03-43ee-8280-ed2819cee95a service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:18.927799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:18.928456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:18.928734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:18.929034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] PATH_INFO: `/v3/users/235c24cb205c47cfaf9befb19095ebc9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:18.930139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=235c24cb205c47cfaf9befb19095ebc9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:18.932087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:18.981265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235c24cb205c47cfaf9befb19095ebc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.981927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235c24cb205c47cfaf9befb19095ebc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:18.989690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235c24cb205c47cfaf9befb19095ebc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.001716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235c24cb205c47cfaf9befb19095ebc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.002350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-48889cef-64ea-4461-8444-8101974aad7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235c24cb205c47cfaf9befb19095ebc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.022132 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3876/7679] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:18 2022] DELETE /identity/v3/users/235c24cb205c47cfaf9befb19095ebc9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:19.295588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.297120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:19.310960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87c441-8205-4dd9-8c51-1f51c6f8435d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.369089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.369744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.370027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.370301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] PATH_INFO: `/v3/projects/9d18b8cbc0fb49b793cada5416f646ea` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.371457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d18b8cbc0fb49b793cada5416f646ea)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.373380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.420384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d18b8cbc0fb49b793cada5416f646ea', 'request_id': 'req-dd918291-db05-4ff5-8841-0d4390fd4680'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.425848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d18b8cbc0fb49b793cada5416f646ea', 'request_id': 'req-dd918291-db05-4ff5-8841-0d4390fd4680'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.432867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d18b8cbc0fb49b793cada5416f646ea', 'request_id': 'req-dd918291-db05-4ff5-8841-0d4390fd4680'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.444342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] The token cache is being invalidate because project 9d18b8cbc0fb49b793cada5416f646ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:19.444631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.445255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-dd918291-db05-4ff5-8841-0d4390fd4680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:19.446868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3804/7680] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:19 2022] DELETE /identity/v3/projects/9d18b8cbc0fb49b793cada5416f646ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:19.452750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.453227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.453323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.473701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:19.474924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:19.515778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:19.536078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3877/7681] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:19.544257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.544555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.544815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.566303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:19.567555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:19.575452 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3805/7682] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:19.579578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.589782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a528a-420c-49fa-9ae3-08b67d0e7d2a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.594278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.594992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.595272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.595536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.596164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:17:19.596456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.598107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.613436 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3878/7683] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:17:19.618233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.627660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb0e4d-d67d-4f76-abe3-9fd6f991e9fd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.633155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.633761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.634037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.634305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.634975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.638837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.668702 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3806/7684] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:17:19 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:17:19.673570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.686956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae5304-b4e9-4dee-a95a-6640e290935b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.753371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.754202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.754487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.754785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.755407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.757548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.781905 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3879/7685] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:17:19 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:17:19.786463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.798751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc738-a007-4c39-a3dd-b0380586a571 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.803988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.804583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.804885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.805156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.805780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.808526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.819945 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3807/7686] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:19.824026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.834603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c221ae8-5ba8-4da0-b7ea-091ee905661e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.839328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.839956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.840237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.840513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.841151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.842785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e8d47c-062e-4d16-b415-ea2ab5984afc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.859227 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3880/7687] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:17:19.860733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:19.872046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a72169-ee3d-493e-8b7f-d11cad37609d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:19.876895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:19.877547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.877878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.878716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] PATH_INFO: `/v3/projects/ba76b0a8f1fe422aa40bd059af063a26/users/15027d4adc2a4435893abef1f3e28e06/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.894343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba76b0a8f1fe422aa40bd059af063a26, user_id=15027d4adc2a4435893abef1f3e28e06, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:19.897125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:19.910620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3808/7688] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:17:19 2022] PUT /identity/v3/projects/ba76b0a8f1fe422aa40bd059af063a26/users/15027d4adc2a4435893abef1f3e28e06/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:17:19.921870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:19.922157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:19.922450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:19.951736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] MFA Rules not processed for user `15027d4adc2a4435893abef1f3e28e06`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:19.952954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:20.051368 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3881/7689] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:20.061816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:20.063704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:20.084884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:20.136422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5fd49-30d7-46d5-a7d3-ec72ef570030 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:20.140401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:20.141116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:20.141497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:20.141925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:20.151456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:20.152908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6575b-b033-4cd4-bf1f-9ac0dc63d99f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:20.162576 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3809/7690] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:17:30.047138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:30.157851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=21 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:36.540796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:36.543230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:36.641561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ce62-cf67-4f9b-982e-cf1834d22b9b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:36.648080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:36.648824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:36.649195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:36.649538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] PATH_INFO: `/v3/users/c1acf8d4b1384c8c8d7014bd8baf897e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:36.653331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1acf8d4b1384c8c8d7014bd8baf897e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:36.654737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:36.741501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1acf8d4b1384c8c8d7014bd8baf897e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:36.742291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1acf8d4b1384c8c8d7014bd8baf897e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:36.765925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1acf8d4b1384c8c8d7014bd8baf897e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:36.785028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1acf8d4b1384c8c8d7014bd8baf897e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:36.790011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1acf8d4b1384c8c8d7014bd8baf897e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:36.814134 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3882/7691] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:36 2022] DELETE /identity/v3/users/c1acf8d4b1384c8c8d7014bd8baf897e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:36.819568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:36.819985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:36.819985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:36.850747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:17:36.852424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:36.927455 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3810/7692] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:17:36.937866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:36.940028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:37.019881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06dcc3-4980-4ed5-b060-120ed72da0a9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:37.025426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:37.026175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:37.026594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:37.026987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:37.038993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:37.040501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:37.053602 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3883/7693] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:17:37.267657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:37.269887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:37.294023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b173-9074-49a0-b662-35fc78e43d5b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:37.306337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:37.307064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:37.307345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:37.307611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] PATH_INFO: `/v3/projects/026ea39b3679444a9c3541e58d2e9f7b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:37.309044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] RBAC: Authorizing `identity:delete_project(project_id=026ea39b3679444a9c3541e58d2e9f7b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:37.311762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:37.380469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026ea39b3679444a9c3541e58d2e9f7b', 'request_id': 'req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.388964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026ea39b3679444a9c3541e58d2e9f7b', 'request_id': 'req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.397164 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026ea39b3679444a9c3541e58d2e9f7b', 'request_id': 'req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.411607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] The token cache is being invalidate because project 026ea39b3679444a9c3541e58d2e9f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:37.411984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.412709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.414532 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3811/7694] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:37 2022] DELETE /identity/v3/projects/026ea39b3679444a9c3541e58d2e9f7b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:37.418339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:37.430167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b445-ca4f-4ba6-a041-caf2b1842164 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:37.460036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:37.513835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:37.514596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:37.514949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:37.515302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] PATH_INFO: `/v3/users/5c5d0cac25164ecba60f1fc493407026` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:37.517294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5d0cac25164ecba60f1fc493407026)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:37.520661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:37.601978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5d0cac25164ecba60f1fc493407026'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.603575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5d0cac25164ecba60f1fc493407026'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.632288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5d0cac25164ecba60f1fc493407026'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.649783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5d0cac25164ecba60f1fc493407026'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.651322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5d0cac25164ecba60f1fc493407026'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:37.667689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3884/7695] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:17:37 2022] DELETE /identity/v3/users/5c5d0cac25164ecba60f1fc493407026 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:37.987959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:37.989405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:38.018677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc7e1e-d636-4c1e-ac3e-bb6c597347db admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:38.100824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:38.101577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:38.101926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:38.103145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] PATH_INFO: `/v3/projects/44a13cc297a446208816af67a5502a15` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:38.104417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a13cc297a446208816af67a5502a15)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:38.106878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:38.147280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:38.170344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a13cc297a446208816af67a5502a15', 'request_id': 'req-4dd775aa-69b0-419d-96c7-fe2eb5115a33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:38.184411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a13cc297a446208816af67a5502a15', 'request_id': 'req-4dd775aa-69b0-419d-96c7-fe2eb5115a33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:38.196508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a13cc297a446208816af67a5502a15', 'request_id': 'req-4dd775aa-69b0-419d-96c7-fe2eb5115a33'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:38.210853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] The token cache is being invalidate because project 44a13cc297a446208816af67a5502a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:17:38.211146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:38.211821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:17:38.213537 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3812/7696] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:17:37 2022] DELETE /identity/v3/projects/44a13cc297a446208816af67a5502a15 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:17:47.664348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:48.207772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:17:51.643072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:17:51.644521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:51.743896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7ec04-41ed-43a6-89b1-c4cee7415455 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:17:51.818851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:17:51.819982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:17:51.819982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:17:51.820345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:17:51.822250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:17:51.839350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:17:51.840844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb9a9e-2d0f-43dc-8b8c-4cfec2bbf025 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:17:51.856481 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3885/7697] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:17:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:18:00.394994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:00.396500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:00.421716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-69b0-419d-96c7-fe2eb5115a33 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:00.427961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:00.428802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:00.429259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:00.429694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:00.433034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:00.450431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:00.452281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:00.468241 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3813/7698] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:18:01.855564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=63 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:18:10.460454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:18:40.652188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49d20-334c-4682-b697-36be4e475a69 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:40.652692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49d20-334c-4682-b697-36be4e475a69 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:40.652782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49d20-334c-4682-b697-36be4e475a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:40.692286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59d49d20-334c-4682-b697-36be4e475a69 None None] MFA Rules not processed for user `82c2160fcae947138ddd7b1b4df1e361`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:18:40.693640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59d49d20-334c-4682-b697-36be4e475a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:40.771744 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3886/7699] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:18:43.336881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:43.338401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:43.415443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977edcc1-1449-41c8-9700-229b1163fe3f service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:43.420034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:43.421044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:43.421541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:43.421820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] PATH_INFO: `/v3/users/15027d4adc2a4435893abef1f3e28e06` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:43.423198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=15027d4adc2a4435893abef1f3e28e06)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:43.425146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:43.487581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15027d4adc2a4435893abef1f3e28e06'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:43.487777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15027d4adc2a4435893abef1f3e28e06'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:43.497247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15027d4adc2a4435893abef1f3e28e06'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:43.507027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15027d4adc2a4435893abef1f3e28e06'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:43.508073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15027d4adc2a4435893abef1f3e28e06'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:43.531396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3814/7700] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:18:43 2022] DELETE /identity/v3/users/15027d4adc2a4435893abef1f3e28e06 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:18:43.535767 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:43.536266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:43.536368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:43.557118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:18:43.558391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:43.647070 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3887/7701] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:18:43.653541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:43.658523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:43.726486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c3221-55f7-49e3-b556-337a0b75f86f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:43.730658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:43.731339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:43.731685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:43.732021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:43.742844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:43.744451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d108b1c-c81a-4e58-bc2a-e9afd415ccb7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:43.756953 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3815/7702] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:18:43.959359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:43.966750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:43.997090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9293ab-2221-4730-80d0-1a4c22a86b1e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.005395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.006317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.006742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.007191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] PATH_INFO: `/v3/projects/ba76b0a8f1fe422aa40bd059af063a26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.010719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba76b0a8f1fe422aa40bd059af063a26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.015277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.100536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba76b0a8f1fe422aa40bd059af063a26', 'request_id': 'req-d50ccd78-2f60-4ba1-8002-0908f72884bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:44.126385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba76b0a8f1fe422aa40bd059af063a26', 'request_id': 'req-d50ccd78-2f60-4ba1-8002-0908f72884bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:44.141211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba76b0a8f1fe422aa40bd059af063a26', 'request_id': 'req-d50ccd78-2f60-4ba1-8002-0908f72884bf'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:44.168046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] The token cache is being invalidate because project ba76b0a8f1fe422aa40bd059af063a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:18:44.168249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:44.170126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d50ccd78-2f60-4ba1-8002-0908f72884bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:18:44.173312 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3888/7703] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:18:43 2022] DELETE /identity/v3/projects/ba76b0a8f1fe422aa40bd059af063a26 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:18:44.178768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.179154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.179154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.236934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:18:44.240096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:44.276601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:18:44.350755 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3816/7704] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:18:44.360334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3540-5902-4b73-82b5-064741502297 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.360811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3540-5902-4b73-82b5-064741502297 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.361102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3540-5902-4b73-82b5-064741502297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.390833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e57e3540-5902-4b73-82b5-064741502297 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:18:44.392076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e57e3540-5902-4b73-82b5-064741502297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:44.402773 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3889/7705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:18:44.407532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.420398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcda52d-5adb-4c83-933a-a982b733fd32 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.426279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.427137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.427602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.427907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.428926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:18:44.429772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.431203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.446350 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3817/7706] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:18:44.450883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3540-5902-4b73-82b5-064741502297 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.463538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3540-5902-4b73-82b5-064741502297 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.470134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.470134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.470409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.471325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.471488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.473409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.516329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:18:44.526818 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3890/7707] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:18:44 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:18:44.529546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.540603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a6ae9-4277-46c4-b0b3-973500d21c41 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.603616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.604227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.604506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.604778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.605414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.607384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.637236 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3818/7708] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:18:44 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:18:44.641589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.653750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c632b4-d694-4c96-9805-197d48278bf8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.659251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.659835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.660111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.660386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.661027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.662655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.673002 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3891/7709] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:18:44.677136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.688525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0731d-8531-491a-9ea3-70461637359b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.693527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.694135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.694430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.694727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.695368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.696964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed31eb-72b5-4719-988f-b8ce5d2d6d73 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.711566 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3819/7710] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:18:44.713007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.724471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cbfd2-ed7f-4901-8225-da93c95f7d0a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.729228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.729881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.730156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.730428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] PATH_INFO: `/v3/projects/5a1d05093abc497c91c2742b8ee917ed/users/e22b4907d18f442b931ebb0be3f1a970/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.746322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1d05093abc497c91c2742b8ee917ed, user_id=e22b4907d18f442b931ebb0be3f1a970, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.750685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:44.765303 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3892/7711] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:18:44 2022] PUT /identity/v3/projects/5a1d05093abc497c91c2742b8ee917ed/users/e22b4907d18f442b931ebb0be3f1a970/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:18:44.776544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.776923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.777204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.805456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] MFA Rules not processed for user `e22b4907d18f442b931ebb0be3f1a970`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:18:44.806804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:44.877179 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3820/7712] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:18:44.884473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:18:44.886466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:18:44.961926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4915c1a-e812-45eb-b0ff-bcdcb9576c7e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:18:44.966314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:18:44.967638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:18:44.968056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:18:44.968436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:18:44.978311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:18:44.979803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62f534-8323-4394-895f-8f6fe62ac2d5 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:18:45.002346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3893/7713] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:18:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:18:54.870394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:18:54.997622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:25.034461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.104704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0386e-cf47-4534-b122-e21489655cfb None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.109861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.110490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.110802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.111080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] PATH_INFO: `/v3/users/e22b4907d18f442b931ebb0be3f1a970` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.112261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22b4907d18f442b931ebb0be3f1a970)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.114204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.162433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22b4907d18f442b931ebb0be3f1a970'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.163144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22b4907d18f442b931ebb0be3f1a970'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.171373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22b4907d18f442b931ebb0be3f1a970'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.179267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22b4907d18f442b931ebb0be3f1a970'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.180083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22b4907d18f442b931ebb0be3f1a970'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.200925 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3821/7714] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:25 2022] DELETE /identity/v3/users/e22b4907d18f442b931ebb0be3f1a970 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:25.205408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.205900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.206013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.226980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:25.228271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:25.287384 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3894/7715] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:25.298268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.299723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:25.359272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bd6c7-67c6-4394-becf-5085470ab174 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.362906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.363493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.363768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.364038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.374429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.375884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6b3d4-3103-4e05-823b-1c762cf49281 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.385235 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3822/7716] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:25.557784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.559881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:25.580333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e97c67-013a-412c-8371-88be402c26a3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.585861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.586579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.586923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.587195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] PATH_INFO: `/v3/projects/5a1d05093abc497c91c2742b8ee917ed` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.588322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1d05093abc497c91c2742b8ee917ed)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.590355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.638801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1d05093abc497c91c2742b8ee917ed', 'request_id': 'req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.645009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1d05093abc497c91c2742b8ee917ed', 'request_id': 'req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.651422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1d05093abc497c91c2742b8ee917ed', 'request_id': 'req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.664360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] The token cache is being invalidate because project 5a1d05093abc497c91c2742b8ee917ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:25.664670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.665278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9e6f4a09-2ffd-4944-b0c0-c4d7024b23ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:25.666811 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3895/7717] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:25 2022] DELETE /identity/v3/projects/5a1d05093abc497c91c2742b8ee917ed => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:25.672193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.672479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.672759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.693553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:25.694705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:25.712043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:25.763079 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3823/7718] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:25.771151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.771910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.771910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.792620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:25.794363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:25.806604 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3896/7719] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:25.811404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.821233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74496e8c-f81f-43f0-9881-b965561a222b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.826048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.826707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.826992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.827267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.827908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:25.828211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.829813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.841332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3824/7720] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:25.845529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.854869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21920231-6ae6-410f-b2a4-a76f707e5cda None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.860468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.861064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.861344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.861618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.862247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.864210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.892215 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3897/7721] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:25 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:25.896395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:25.906914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f7f5-92a2-4cc3-83e2-55e2a6b1bb68 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:25.962933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:25.963455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:25.963735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:25.964010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:25.964659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:25.967187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:25.989006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3825/7722] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:25 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:25.992658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:26.003648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f74f-47f7-424f-b613-a3ac81dbae77 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:26.008574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:26.009178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:26.009462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:26.009736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:26.010375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:26.012050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:26.022081 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3898/7723] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:26.026259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:26.037048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9044f8-8ba6-47b8-ad29-4b87c762f494 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:26.041665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:26.042288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:26.042608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:26.043569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:26.044595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:26.046201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e87c21-5bb8-4c19-baf1-597d03b035cc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:26.057843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3826/7724] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:26.061921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:26.071978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b8ee5-7ce9-482c-b3d5-19e84ff3dcf1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:26.076736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:26.077371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:26.077652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:26.077925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] PATH_INFO: `/v3/projects/26ad1726138b4a3da338a2f254ebdbee/users/0e8775437bce466386e210886b85c922/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:26.095695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ad1726138b4a3da338a2f254ebdbee, user_id=0e8775437bce466386e210886b85c922, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:26.098613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:26.113970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3899/7725] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:26 2022] PUT /identity/v3/projects/26ad1726138b4a3da338a2f254ebdbee/users/0e8775437bce466386e210886b85c922/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:19:26.124638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:26.124952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:26.125230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:26.155302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] MFA Rules not processed for user `0e8775437bce466386e210886b85c922`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:26.156574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:26.224712 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3827/7726] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:26.235326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:26.237043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:26.260299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:26.304505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46856bb5-9603-4631-9187-b11ae42d1365 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:26.312466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:26.313083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:26.313366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:26.313644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:26.322661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:26.324094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af529b5-6341-4091-ab42-b801605fb769 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:26.333376 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3900/7727] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:36.221496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:36.329796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=17 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:42.925525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:42.988453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efafc97-f18d-4199-8bfc-c613b0086506 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:42.993013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:42.993631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:42.993910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:42.994183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] PATH_INFO: `/v3/users/0e8775437bce466386e210886b85c922` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:42.995449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8775437bce466386e210886b85c922)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:42.997562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.042594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8775437bce466386e210886b85c922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.043258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8775437bce466386e210886b85c922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.050588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8775437bce466386e210886b85c922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.057488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8775437bce466386e210886b85c922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.058457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8775437bce466386e210886b85c922'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.076016 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3828/7728] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:42 2022] DELETE /identity/v3/users/0e8775437bce466386e210886b85c922 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:43.080539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.081001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.081125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.100584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:43.101781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:43.167852 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3901/7729] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:43.177520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.179137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:43.238894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11313714-f612-4ebf-beb8-61c0d6fb85e3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.242611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.243205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.243482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.243752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.253170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.254667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accfcf9-8885-4d96-b335-b79a7d2d99bc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.265070 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3829/7730] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:43.447634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.449892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:43.463969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95361991-70b2-4fdd-a7fe-f490011cc6b3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.469598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.470322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.470665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.470996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] PATH_INFO: `/v3/projects/26ad1726138b4a3da338a2f254ebdbee` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.472560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ad1726138b4a3da338a2f254ebdbee)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.474890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.522892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ad1726138b4a3da338a2f254ebdbee', 'request_id': 'req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.529272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ad1726138b4a3da338a2f254ebdbee', 'request_id': 'req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.535451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ad1726138b4a3da338a2f254ebdbee', 'request_id': 'req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.547593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] The token cache is being invalidate because project 26ad1726138b4a3da338a2f254ebdbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:43.547883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.548512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ab8df59-0e61-42ab-bde5-f54e008dcb0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:43.550045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3902/7731] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:43 2022] DELETE /identity/v3/projects/26ad1726138b4a3da338a2f254ebdbee => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:43.554872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d419-833b-4c39-8985-d323d6938aed None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.555158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d419-833b-4c39-8985-d323d6938aed None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.555434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080d419-833b-4c39-8985-d323d6938aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.576046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7080d419-833b-4c39-8985-d323d6938aed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:43.577272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7080d419-833b-4c39-8985-d323d6938aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:43.594360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:43.640439 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3830/7732] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:43.648516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.648976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.649282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.669973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:43.671206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:43.679067 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3903/7733] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:43.683873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d419-833b-4c39-8985-d323d6938aed None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.693425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080d419-833b-4c39-8985-d323d6938aed None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.698000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.698669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.698943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.699210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.699864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:43.700162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.701752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.713320 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3831/7734] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:43.718030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.727930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1eace3-1ded-494b-a7ef-d692cba3098a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.733802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.734404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.734718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.734987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.735619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.737609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.765802 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3904/7735] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:43.770189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.780568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d8e5a-f476-4f9a-b6ac-e24ada739945 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.838047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.838687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.839156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.839507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.840146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.842074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.864475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3832/7736] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:43.868750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.879774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717341f-0e6a-4624-8ed3-28543ceea636 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.884373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.884960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.885236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.885504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.886429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.888041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.897691 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3905/7737] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:43.902136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.912187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a822-eea0-42a3-b162-c5a715d911dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.917455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.918112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.918319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.918623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.919254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.920851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376d96e-65e0-4509-a4c6-b31b8c909254 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.931401 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3833/7738] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:43.936048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:43.945928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f487754-f141-45ad-93a5-70bbb10ad9ca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:43.950631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:43.951268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.951545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.951817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] PATH_INFO: `/v3/projects/d0376aad31fb4f19bfab05777174a5f7/users/eda7922b483e4aa9854f6fac24e9583c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:43.966621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0376aad31fb4f19bfab05777174a5f7, user_id=eda7922b483e4aa9854f6fac24e9583c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:43.969253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:43.982731 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3906/7739] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:43 2022] PUT /identity/v3/projects/d0376aad31fb4f19bfab05777174a5f7/users/eda7922b483e4aa9854f6fac24e9583c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:43.992900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:43.993217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:43.993512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.021046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] MFA Rules not processed for user `eda7922b483e4aa9854f6fac24e9583c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:44.022256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.086514 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3834/7740] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:44.103614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:44.106267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.128274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:44.171053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb89cbba-9288-459f-a99c-17c02cefa894 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:44.174744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:44.175348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.175625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.175894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.185262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:44.186742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e942a5a-a779-4bb7-93c9-0d817374b4d3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:44.196265 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3907/7741] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:44.210228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:44.229241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:44.275522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652288d-70a9-4254-a204-d638a18e0e55 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:44.280394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:44.281007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.281281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.281551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] PATH_INFO: `/v3/users/eda7922b483e4aa9854f6fac24e9583c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.282708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda7922b483e4aa9854f6fac24e9583c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:44.284617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:44.330394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda7922b483e4aa9854f6fac24e9583c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.331106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda7922b483e4aa9854f6fac24e9583c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.338170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda7922b483e4aa9854f6fac24e9583c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.345278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda7922b483e4aa9854f6fac24e9583c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.345948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda7922b483e4aa9854f6fac24e9583c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.363893 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3835/7742] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:44 2022] DELETE /identity/v3/users/eda7922b483e4aa9854f6fac24e9583c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:44.368318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.368601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.368879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.390590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:44.391839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.452814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3908/7743] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:44.462754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:44.464836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.524684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945056a-bbcf-4814-a855-8ebb4d3bcf61 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:44.529042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:44.529636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.529936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.530209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.539189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:44.540733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7259dea-89f9-4e42-aedc-0347f723ff11 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:44.550929 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3836/7744] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:44.822397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:44.824735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.836953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f74e0-0024-421e-9930-c31d496006ed None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:44.841961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:44.842567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.842856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.843163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] PATH_INFO: `/v3/projects/d0376aad31fb4f19bfab05777174a5f7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.844392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0376aad31fb4f19bfab05777174a5f7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:44.846299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:44.893452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0376aad31fb4f19bfab05777174a5f7', 'request_id': 'req-343372f0-c5b1-40c3-9830-ab17bbdff600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.899095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0376aad31fb4f19bfab05777174a5f7', 'request_id': 'req-343372f0-c5b1-40c3-9830-ab17bbdff600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.905453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0376aad31fb4f19bfab05777174a5f7', 'request_id': 'req-343372f0-c5b1-40c3-9830-ab17bbdff600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.917927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] The token cache is being invalidate because project d0376aad31fb4f19bfab05777174a5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:44.918216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.918877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-343372f0-c5b1-40c3-9830-ab17bbdff600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:44.920375 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3909/7745] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:44 2022] DELETE /identity/v3/projects/d0376aad31fb4f19bfab05777174a5f7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:44.925569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:44.925850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:44.926127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:44.945881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:44.947092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:44.981277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:45.007580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3837/7746] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:45.015962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.016249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.016554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.036276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:45.037480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:45.045688 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3910/7747] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:45.049935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.060281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8670ed-8086-4dbd-93c5-fbeb17051f8a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.064879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.065464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.065743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.066014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.066685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:45.066991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.068612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.086705 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3838/7748] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:45.091104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.100541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5373a04-9db3-4763-a24a-45c4ee939e8f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.105947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.106581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.106845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.107119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.107757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.109657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.136980 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3911/7749] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:45.141068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.151512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e9815-a0da-4ba8-a6b8-0c0a44590796 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.210889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.211488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.211770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.212045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.212660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.214647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.236139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3839/7750] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:45.240292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.251244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0851314-f4b7-4063-b384-9d8043f6edc4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.256244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.256411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.256688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.256960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.257635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.259262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.269296 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3912/7751] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:45.273452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.283548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53b42-6ec9-4b24-be22-d24eb9ed338a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.288223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.288811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.289089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.289365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.289994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.291625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393948ef-1ca1-4e87-970b-0b5dbbe36dc7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.302719 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3840/7752] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:45.306713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.317401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31633249-2182-4234-ae2a-3e950100ed27 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.321994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.322662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.322947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.323494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] PATH_INFO: `/v3/projects/09b730afd60d461cbce74e596e2a2fbb/users/e15dc0a533ad4c6fae32de36a9e6ac62/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.325561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:45.340072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b730afd60d461cbce74e596e2a2fbb, user_id=e15dc0a533ad4c6fae32de36a9e6ac62, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.342744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.356891 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3913/7753] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:45 2022] PUT /identity/v3/projects/09b730afd60d461cbce74e596e2a2fbb/users/e15dc0a533ad4c6fae32de36a9e6ac62/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:19:45.367924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.368214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.368498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.396859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] MFA Rules not processed for user `e15dc0a533ad4c6fae32de36a9e6ac62`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:45.398339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:45.462805 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3841/7754] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:45.469369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.536253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba74e-aeb9-471c-89c4-723f045b8625 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.540997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.541598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.542525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.542867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.543650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.545536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.574480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3914/7755] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:45.578190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.590078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7ff7d-7683-4cdc-b811-a8fb45c60544 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.626054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:45.656342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.656939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.657214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.657484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.658101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.660054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.681844 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3842/7756] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/users => generated 308 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:45.686095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.696247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b19d1-a329-4f3f-aace-57b117f79f08 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.701439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.702028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.702304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.702615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.703236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.704811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.717526 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3915/7757] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:45.721641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.732369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f985e-0079-4fd2-9352-3ab976f2fd71 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.736969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.737601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.737877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.738149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] PATH_INFO: `/v3/projects/9b7de316402f434a96406da2aadccaf8/users/38733a058d044818ae236e138d73eb59/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.752981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7de316402f434a96406da2aadccaf8, user_id=38733a058d044818ae236e138d73eb59, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.755690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.769408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3843/7758] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:45 2022] PUT /identity/v3/projects/9b7de316402f434a96406da2aadccaf8/users/38733a058d044818ae236e138d73eb59/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:45.773498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.784553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2a83d-4814-4198-adb2-902eb27c89cf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.843262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.843853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.844128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.844393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.845006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.846639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ae059-6bae-4c1c-bdcf-ac7c2760871c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.856967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3916/7759] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:19:45.861392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:45.871711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013cbc2-6225-45d5-b09d-57d9a6427754 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:45.876312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:45.876946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.877221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.877488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] PATH_INFO: `/v3/projects/9b7de316402f434a96406da2aadccaf8/users/38733a058d044818ae236e138d73eb59/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.879429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7de316402f434a96406da2aadccaf8, user_id=38733a058d044818ae236e138d73eb59, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:45.882059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:45.892627 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3844/7760] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:45 2022] PUT /identity/v3/projects/9b7de316402f434a96406da2aadccaf8/users/38733a058d044818ae236e138d73eb59/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:45.902999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:45.903284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:45.903556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:45.932394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] MFA Rules not processed for user `38733a058d044818ae236e138d73eb59`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:45.933946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:45.976019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:46.002875 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3917/7761] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:46.014864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:46.016648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:46.091477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375d99a-85d1-45d9-9522-eb348da8d6d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:46.095214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:46.095808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:46.096088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:46.096361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:46.105409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:46.106882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2be71-dfa5-4330-be36-80a87d45ffe4 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:46.116722 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3845/7762] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:46 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:46.190342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:46.252604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8405692-4ee2-4081-8904-1abc1c41b7b5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:46.257526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:46.258154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:46.258471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:46.258776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] PATH_INFO: `/v3/users/e15dc0a533ad4c6fae32de36a9e6ac62` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:46.259931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15dc0a533ad4c6fae32de36a9e6ac62)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:46.262536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:46.310294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15dc0a533ad4c6fae32de36a9e6ac62'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.311049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15dc0a533ad4c6fae32de36a9e6ac62'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.318336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15dc0a533ad4c6fae32de36a9e6ac62'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.330769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15dc0a533ad4c6fae32de36a9e6ac62'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.331394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15dc0a533ad4c6fae32de36a9e6ac62'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.343232 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3918/7763] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:46 2022] DELETE /identity/v3/users/e15dc0a533ad4c6fae32de36a9e6ac62 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:46.347641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:46.347925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:46.348197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:46.367010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:46.368204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:46.431233 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3846/7764] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:46.441411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:46.442970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:46.502352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a41ed4-458f-4b47-ba9d-13ad68f6b7e4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:46.506078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:46.506754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:46.507037 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:46.507310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:46.517074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:46.518557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:46.523785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:46.529028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3919/7765] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:46.804928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:46.806369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:46.825737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8e312-ea6f-4ed2-95c7-37641b9d837c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:46.831499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:46.832151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:46.832425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:46.832695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] PATH_INFO: `/v3/projects/09b730afd60d461cbce74e596e2a2fbb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:46.833881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b730afd60d461cbce74e596e2a2fbb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:46.835883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:46.854012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:46.891581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b730afd60d461cbce74e596e2a2fbb', 'request_id': 'req-46cbc241-cdc3-414d-99fd-f466c3e88257'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.897594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b730afd60d461cbce74e596e2a2fbb', 'request_id': 'req-46cbc241-cdc3-414d-99fd-f466c3e88257'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.906204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b730afd60d461cbce74e596e2a2fbb', 'request_id': 'req-46cbc241-cdc3-414d-99fd-f466c3e88257'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.920404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] The token cache is being invalidate because project 09b730afd60d461cbce74e596e2a2fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:46.920682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.921379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:46.922996 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3847/7766] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:46 2022] DELETE /identity/v3/projects/09b730afd60d461cbce74e596e2a2fbb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:46.927164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:46.938401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415590c-9ef1-4ae8-b25c-975c56cc88e3 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:46.999479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.000112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.000393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.000666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] PATH_INFO: `/v3/users/38733a058d044818ae236e138d73eb59` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.001783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=38733a058d044818ae236e138d73eb59)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.003756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.050118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38733a058d044818ae236e138d73eb59'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.050789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38733a058d044818ae236e138d73eb59'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.057954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38733a058d044818ae236e138d73eb59'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.069917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38733a058d044818ae236e138d73eb59'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.070622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-decbf3b3-e9c3-43aa-942a-dbfd7532e21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38733a058d044818ae236e138d73eb59'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.083028 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3920/7767] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:46 2022] DELETE /identity/v3/users/38733a058d044818ae236e138d73eb59 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:47.359573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.361021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:47.380817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbc241-cdc3-414d-99fd-f466c3e88257 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.439325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.439868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.440147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.440419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] PATH_INFO: `/v3/projects/9b7de316402f434a96406da2aadccaf8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.441523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7de316402f434a96406da2aadccaf8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.443480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.493461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7de316402f434a96406da2aadccaf8', 'request_id': 'req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.499030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7de316402f434a96406da2aadccaf8', 'request_id': 'req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.506222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7de316402f434a96406da2aadccaf8', 'request_id': 'req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.518470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] The token cache is being invalidate because project 9b7de316402f434a96406da2aadccaf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:47.518783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.519431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9cf98ba7-89d8-444a-9d16-6ac3d8e06460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:47.520882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3848/7768] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:47 2022] DELETE /identity/v3/projects/9b7de316402f434a96406da2aadccaf8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:47.526006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.527073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.527382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.547107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:47.548303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:47.610317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3921/7769] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:47.617702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027e92-5874-495b-9577-3f131f415433 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.617987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027e92-5874-495b-9577-3f131f415433 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.618256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14027e92-5874-495b-9577-3f131f415433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.642225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-14027e92-5874-495b-9577-3f131f415433 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:47.643472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-14027e92-5874-495b-9577-3f131f415433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:47.652368 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3849/7770] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:47.656536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.666143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c2b5-51b9-43ed-a3ce-ee36d88d7438 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.671454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.672043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.672320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.672588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.673216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:47.673514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.675146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.687200 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3922/7771] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:47.691466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14027e92-5874-495b-9577-3f131f415433 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.701746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14027e92-5874-495b-9577-3f131f415433 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.706537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.707123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.707405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.707679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.708311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.710236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.737728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3850/7772] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:47 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:47.741737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.751617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa380b-01e6-436a-bd09-efd9fa16656d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.755456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:47.819029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.819686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.819955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.820230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.821057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.823050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.846469 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3923/7773] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:47 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:47.850841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.858999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:47.863497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f560d9-5143-47e9-98df-427cf5dff26e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.870352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.871283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.871859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.872436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.873297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.875083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.885903 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3851/7774] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:47.890163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.901846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f14e9e-cd15-43d2-b633-59ae39b2645f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.907202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.907827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.908104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.908374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.909022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.911693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177974a-ebd9-4c8e-a98a-eb5d833a807e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.928181 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3924/7775] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:47.929561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:47.940363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365908f-eee1-4b37-813c-a9da3098fc2c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:47.945194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:47.945794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.946071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.946338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] PATH_INFO: `/v3/projects/ca7b3d2abd844c49976b6b486a52110e/users/4e0af2628152463fb770f3d5887a1c40/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:47.962850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7b3d2abd844c49976b6b486a52110e, user_id=4e0af2628152463fb770f3d5887a1c40, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:47.966108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:47.981276 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3852/7776] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:47 2022] PUT /identity/v3/projects/ca7b3d2abd844c49976b6b486a52110e/users/4e0af2628152463fb770f3d5887a1c40/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:47.992414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:47.992723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:47.992999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.023181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] MFA Rules not processed for user `4e0af2628152463fb770f3d5887a1c40`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:48.024431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:48.096752 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3925/7777] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:48.101393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.169094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a7668-74ba-4525-8b7c-335b9002b658 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.174168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.174799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.175085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.175360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.175996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.177951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.206685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3853/7778] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:48 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:48.210562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.221691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583a482-4c64-485a-bae3-d4fa4ffdf2cb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.277710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.278320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.278646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.278926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.279561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.281503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.302190 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3926/7779] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:48 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:48.306090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.317065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5bdbc-5fd4-4192-986f-63524c832a6c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.321725 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.322319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.322637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.322913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.323659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.325255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.335841 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3854/7780] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:48.342165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.348629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:48.352878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586575a-5df4-4b1d-b6d9-63b114044ea8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.359001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.359685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.359993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.360264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] PATH_INFO: `/v3/projects/76bfe70a37de4acf857d0ff0d385032c/users/0fdee45bc495490cac2a169e0ebccfc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.376575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bfe70a37de4acf857d0ff0d385032c, user_id=0fdee45bc495490cac2a169e0ebccfc9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.379359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.393206 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3927/7781] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:48 2022] PUT /identity/v3/projects/76bfe70a37de4acf857d0ff0d385032c/users/0fdee45bc495490cac2a169e0ebccfc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:48.397206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.411240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354bc76-9bd3-4d5a-93fb-bc65cbee3dff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.473214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.473764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.474045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.474321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.474996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.476617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116f47-e315-45ca-b4de-d9f80e0124f1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.487984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3855/7782] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:48.492025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.502323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216caa-51e0-47c6-98ba-e1b07bf4adbd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.507063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.507716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.507999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.508274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] PATH_INFO: `/v3/projects/76bfe70a37de4acf857d0ff0d385032c/users/0fdee45bc495490cac2a169e0ebccfc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.510219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bfe70a37de4acf857d0ff0d385032c, user_id=0fdee45bc495490cac2a169e0ebccfc9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.512917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.524606 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3928/7783] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:48 2022] PUT /identity/v3/projects/76bfe70a37de4acf857d0ff0d385032c/users/0fdee45bc495490cac2a169e0ebccfc9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:48.535616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.535945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.536410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.567588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] MFA Rules not processed for user `0fdee45bc495490cac2a169e0ebccfc9`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:48.568803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:48.600552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:48.647386 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3856/7784] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:19:48.661516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.664509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:48.728871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fde5f-6a93-48be-a3ca-ce646d5e912d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.733242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.733849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.734128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.734398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.743886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.745366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934a896-8d40-419b-bcce-ce6961a5b706 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:48.755377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3929/7785] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:48.909121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:48.973832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635847-4841-4d79-aa05-8e964e255c76 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:48.978409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:48.979221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:48.979345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:48.979610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] PATH_INFO: `/v3/users/4e0af2628152463fb770f3d5887a1c40` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:48.980831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0af2628152463fb770f3d5887a1c40)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:48.982810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:49.028801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0af2628152463fb770f3d5887a1c40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.029450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0af2628152463fb770f3d5887a1c40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.036738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0af2628152463fb770f3d5887a1c40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.043733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0af2628152463fb770f3d5887a1c40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.044373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0af2628152463fb770f3d5887a1c40'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.061528 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3857/7786] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:48 2022] DELETE /identity/v3/users/4e0af2628152463fb770f3d5887a1c40 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:49.066117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:49.066405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:49.066788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:49.085943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:49.087144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:49.144158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3930/7787] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:49.154871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:49.156369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:49.202313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:49.218094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea1c8-e546-462e-8a28-04594ae04ef3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:49.221992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:49.222626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:49.222910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:49.223183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:49.232638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:49.234019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:49.244359 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3858/7788] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:49.257733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:49.270766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37b43c-45eb-46c2-beca-5a67cf067550 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:49.274661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:49.275336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:49.275940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:49.276395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:49.278240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:49.356679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:49.386956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:49.388396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:49.403146 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3931/7789] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:49.541927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:49.543484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:49.565176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627234a0-53e8-4caa-8cc0-0b0eeded44ad service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:49.570429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:49.571000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:49.571281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:49.571555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] PATH_INFO: `/v3/projects/ca7b3d2abd844c49976b6b486a52110e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:49.572662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7b3d2abd844c49976b6b486a52110e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:49.575279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:49.623655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7b3d2abd844c49976b6b486a52110e', 'request_id': 'req-f5859a23-e09b-421c-93ed-5a626031647e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.630251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7b3d2abd844c49976b6b486a52110e', 'request_id': 'req-f5859a23-e09b-421c-93ed-5a626031647e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.638464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7b3d2abd844c49976b6b486a52110e', 'request_id': 'req-f5859a23-e09b-421c-93ed-5a626031647e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.650217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] The token cache is being invalidate because project ca7b3d2abd844c49976b6b486a52110e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:49.650519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.651207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.652864 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3859/7790] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:49 2022] DELETE /identity/v3/projects/ca7b3d2abd844c49976b6b486a52110e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:49.656827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:49.667815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74985107-7f9b-4738-af8d-b851c8033146 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:49.724060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:49.724686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:49.724967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:49.725239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] PATH_INFO: `/v3/users/0fdee45bc495490cac2a169e0ebccfc9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:49.726350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdee45bc495490cac2a169e0ebccfc9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:49.728301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:49.774797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdee45bc495490cac2a169e0ebccfc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.775377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdee45bc495490cac2a169e0ebccfc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.782317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdee45bc495490cac2a169e0ebccfc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.794637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdee45bc495490cac2a169e0ebccfc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.795277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9be77dd4-719f-414b-978f-65a18eb445f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdee45bc495490cac2a169e0ebccfc9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:49.807135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3932/7791] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:49 2022] DELETE /identity/v3/users/0fdee45bc495490cac2a169e0ebccfc9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:50.074763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.077534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:50.098918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5859a23-e09b-421c-93ed-5a626031647e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.158380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.159048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.159327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.159597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] PATH_INFO: `/v3/projects/76bfe70a37de4acf857d0ff0d385032c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.160707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76bfe70a37de4acf857d0ff0d385032c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.162645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.208250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76bfe70a37de4acf857d0ff0d385032c', 'request_id': 'req-ac5b9621-4176-46a9-a79b-caca16737b11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:50.214318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76bfe70a37de4acf857d0ff0d385032c', 'request_id': 'req-ac5b9621-4176-46a9-a79b-caca16737b11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:50.222029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76bfe70a37de4acf857d0ff0d385032c', 'request_id': 'req-ac5b9621-4176-46a9-a79b-caca16737b11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:50.230212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:50.234262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] The token cache is being invalidate because project 76bfe70a37de4acf857d0ff0d385032c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:50.234986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:50.235699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac5b9621-4176-46a9-a79b-caca16737b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:50.237205 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3860/7792] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:50 2022] DELETE /identity/v3/projects/76bfe70a37de4acf857d0ff0d385032c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:50.242211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.242536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.242817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.262258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:50.263506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:50.313271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:50.324270 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3933/7793] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:50.331706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.331996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.332277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.352917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:50.354818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:50.362801 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3861/7794] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:50.366988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.377177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6d1b0-6bc4-4f41-9e71-0354d2e6c7df None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.381869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.382463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.382780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.383052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.383718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:50.384020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.386144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.397777 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3934/7795] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:50.403238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.412602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041e8fc-45bb-4182-94a0-bce431b284e9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.417719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.418311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.418637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.418913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.419662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.421676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.449831 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3862/7796] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:50.454035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.464994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212193f-3e33-4364-9fdc-809cb513f75c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.521828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.522445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.522760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.523300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.523929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.525839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.549799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3935/7797] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:50.553912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.565155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262798-d116-4b18-8334-881880eaf356 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.569880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.570481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.570812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.574180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.574180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.574180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.586571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3863/7798] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:50.588395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.599126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c993120-78c5-473c-b798-71db0c01979d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.603817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.604479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.604832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.605148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.605797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.607418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e14ee2-47b0-4ffd-bb06-608513516d55 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.617974 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3936/7799] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:50.622546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.633046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e96933-1985-423d-8bed-b29973ad878a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.639025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.639685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.640359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.640637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] PATH_INFO: `/v3/projects/e4d16ccbeab045a08cf016d821eb3d18/users/cddee685c64b4eb496cd3a7b914e08fa/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.656207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d16ccbeab045a08cf016d821eb3d18, user_id=cddee685c64b4eb496cd3a7b914e08fa, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.658898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.673163 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3864/7800] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:50 2022] PUT /identity/v3/projects/e4d16ccbeab045a08cf016d821eb3d18/users/cddee685c64b4eb496cd3a7b914e08fa/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:50.684546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.685067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.685189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.717546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] MFA Rules not processed for user `cddee685c64b4eb496cd3a7b914e08fa`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:50.718789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:50.782354 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3937/7801] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:50.786876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.848953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3a565-de07-4a24-94bb-55f9ce1c489b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.853536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.854826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.855308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.855693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.856308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.858329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.884131 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3865/7802] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:50.888430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.900667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68079774-2f0c-4685-8133-02edc69b5bda None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:50.947514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:50.958239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:50.958877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:50.959160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:50.959429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:50.960086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:50.962012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:50.984399 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3938/7803] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:50 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:50.988366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:50.999005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057a61d-3758-44f7-ac7a-95c96b3bffb2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.003605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.004190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.004464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.004733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.005380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.006998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.018919 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3866/7804] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:19:51.023657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:51.034250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ea682-81f6-4aee-8ddc-97a9e154051a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.038875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.039511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.039789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.040062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] PATH_INFO: `/v3/projects/fd34a167a2cb411da1639c32a500c2ce/users/0020499ca5b14f92bb878c2d129bb584/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.055012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd34a167a2cb411da1639c32a500c2ce, user_id=0020499ca5b14f92bb878c2d129bb584, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.058031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.070349 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3939/7805] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:51 2022] PUT /identity/v3/projects/fd34a167a2cb411da1639c32a500c2ce/users/0020499ca5b14f92bb878c2d129bb584/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:51.074704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:51.084783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7119fa6-a12f-4d4e-838c-1902acb7456d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.141872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.142408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.142696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.142966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.143590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.145169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafc10c-6a05-48a2-a212-213e42882b73 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.156838 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3867/7806] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:51.161051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:51.171552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fee37-07fa-427a-931b-76814f4ac1a1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.176443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.177082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.177358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.177630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] PATH_INFO: `/v3/projects/fd34a167a2cb411da1639c32a500c2ce/users/0020499ca5b14f92bb878c2d129bb584/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.179561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd34a167a2cb411da1639c32a500c2ce, user_id=0020499ca5b14f92bb878c2d129bb584, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.182198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.193236 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3940/7807] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:51 2022] PUT /identity/v3/projects/fd34a167a2cb411da1639c32a500c2ce/users/0020499ca5b14f92bb878c2d129bb584/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:51.204026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.204296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.204573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.219363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:51.234228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] MFA Rules not processed for user `0020499ca5b14f92bb878c2d129bb584`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:51.235460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:51.301902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3868/7808] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:51.313333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:51.315114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:51.375081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb85e0-1758-4313-93cf-e6cb8185849e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.378824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.379420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.379700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.379971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.389539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.391005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.400668 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3941/7809] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:19:51.447451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:51.459925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d023fa2-4e48-4017-8132-61a001a9c984 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:51.464303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:51.465026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:51.465432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:51.465705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:51.521787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:51.523542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56648a79-63ae-46a4-ad26-65faf2f5086e service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:51.532484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3869/7810] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:53.173303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:53.235775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ce501-94c2-4617-9bfb-d5231b6bd0e0 service cinder] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:53.240394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:53.241019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:53.241301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:53.241579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] PATH_INFO: `/v3/users/cddee685c64b4eb496cd3a7b914e08fa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:53.242736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cddee685c64b4eb496cd3a7b914e08fa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:53.245194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:53.292016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cddee685c64b4eb496cd3a7b914e08fa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.292702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cddee685c64b4eb496cd3a7b914e08fa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.295246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:53.300612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cddee685c64b4eb496cd3a7b914e08fa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.313945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cddee685c64b4eb496cd3a7b914e08fa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.314621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cddee685c64b4eb496cd3a7b914e08fa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.327542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3942/7811] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:53 2022] DELETE /identity/v3/users/cddee685c64b4eb496cd3a7b914e08fa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:53.332069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:53.332552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:53.332653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:53.351762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:53.352964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:53.415097 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3870/7812] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:53.425645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:53.427077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:53.489267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd615a8-e288-4473-93fc-43d6a2df5a33 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:53.492993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:53.493594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:53.493875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:53.494146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:53.503846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:53.505285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:53.515179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3943/7813] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:53.790941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:53.793001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:53.815845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0817c3-eedc-4b28-ab74-fc802d56ad69 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:53.820605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:53.821127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:53.821402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:53.821667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] PATH_INFO: `/v3/projects/e4d16ccbeab045a08cf016d821eb3d18` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:53.822817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d16ccbeab045a08cf016d821eb3d18)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:53.824758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:53.859683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:53.879476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d16ccbeab045a08cf016d821eb3d18', 'request_id': 'req-a2172fd8-efb0-4b61-82fd-b759860af3db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.884995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d16ccbeab045a08cf016d821eb3d18', 'request_id': 'req-a2172fd8-efb0-4b61-82fd-b759860af3db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.893666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d16ccbeab045a08cf016d821eb3d18', 'request_id': 'req-a2172fd8-efb0-4b61-82fd-b759860af3db'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.906266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] The token cache is being invalidate because project e4d16ccbeab045a08cf016d821eb3d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:53.906710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.907405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:53.909158 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3871/7814] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:53 2022] DELETE /identity/v3/projects/e4d16ccbeab045a08cf016d821eb3d18 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:53.913061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:53.924292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3829c2-41da-426f-9bf7-f05b34a70bd4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:53.980880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:53.981430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:53.981708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:53.982026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] PATH_INFO: `/v3/users/0020499ca5b14f92bb878c2d129bb584` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:53.983258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0020499ca5b14f92bb878c2d129bb584)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:53.985207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.031581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0020499ca5b14f92bb878c2d129bb584'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.032247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0020499ca5b14f92bb878c2d129bb584'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.039550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0020499ca5b14f92bb878c2d129bb584'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.051904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0020499ca5b14f92bb878c2d129bb584'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.052527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-593a74ef-0a1e-4ae5-a3fa-af120332ad7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0020499ca5b14f92bb878c2d129bb584'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.053668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:54.067103 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3944/7815] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:53 2022] DELETE /identity/v3/users/0020499ca5b14f92bb878c2d129bb584 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:54.335788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.337261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:54.353248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2172fd8-efb0-4b61-82fd-b759860af3db admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.412839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.413470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.413748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.414018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] PATH_INFO: `/v3/projects/fd34a167a2cb411da1639c32a500c2ce` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.415224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd34a167a2cb411da1639c32a500c2ce)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.417800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.463011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd34a167a2cb411da1639c32a500c2ce', 'request_id': 'req-445b6c08-e944-451c-aa11-12775ef4c3b3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.468947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd34a167a2cb411da1639c32a500c2ce', 'request_id': 'req-445b6c08-e944-451c-aa11-12775ef4c3b3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.476378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd34a167a2cb411da1639c32a500c2ce', 'request_id': 'req-445b6c08-e944-451c-aa11-12775ef4c3b3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.488163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] The token cache is being invalidate because project fd34a167a2cb411da1639c32a500c2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:54.488478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.489100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-445b6c08-e944-451c-aa11-12775ef4c3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:54.490588 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3872/7816] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:54 2022] DELETE /identity/v3/projects/fd34a167a2cb411da1639c32a500c2ce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:54.500000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.500486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.500585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.519968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:54.521101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:54.580583 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3945/7817] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:54.588466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.589261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.589531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.609403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:54.610627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:54.618453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3873/7818] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:54.622915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.633034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5f8f0-1f6b-45e2-b143-7418d2a2f54d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.638167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.638939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.639217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.639486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.640121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:19:54.640425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.642011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.655000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3946/7819] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:19:54.659279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.669472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46715d8-883b-4a4a-93b8-b7fe2edacf8c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.674105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.674728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.675004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.675268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.675895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.677794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.705209 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3874/7820] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:54 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:54.709474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.720140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8375-8a58-4796-a926-68eb42edb5d5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.775832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.776981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.777433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.777716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.778339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.780283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.801847 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3947/7821] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:54 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:54.806165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.816557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ec184-a0c3-406e-976d-19dd5e426ba2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.821234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.821822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.822104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.822376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.823091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.824702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.834480 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3875/7822] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:54.838462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.849405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f97c8b-6b24-406a-8f11-8df309fa2d8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.854129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.855568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.855814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.856076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.856710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.858298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ca867-5f7b-423d-8390-51020f7b3b70 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.868829 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3948/7823] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:54.873132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:54.879607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:54.883511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040754-8540-401a-9be7-48b612d284c5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:54.891035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:54.891766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.892188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.892516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] PATH_INFO: `/v3/projects/40d1d406f913492ab44d4cd5e1c97852/users/1c22c8325d7941589aad7e561caf1e71/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.909393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d1d406f913492ab44d4cd5e1c97852, user_id=1c22c8325d7941589aad7e561caf1e71, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:54.912061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:54.924731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3876/7824] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:54 2022] PUT /identity/v3/projects/40d1d406f913492ab44d4cd5e1c97852/users/1c22c8325d7941589aad7e561caf1e71/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:54.936249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:54.936535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:54.936807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:54.966811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] MFA Rules not processed for user `1c22c8325d7941589aad7e561caf1e71`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:54.968072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:55.012507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:55.036289 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3949/7825] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:55.040777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.103716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9464a53-bea1-4ebe-87da-d1d1d50b8cc2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.108943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.109547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.109826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.110103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.110753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.112656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.139185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3877/7826] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:19:55 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:19:55.143227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.154869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52393beb-653e-41a9-90dd-b44b880db9fb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.209518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.210114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.210395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.210704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.211337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.213279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.234730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3950/7827] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:19:55 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:19:55.238478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.249385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41accac0-d533-43cc-81b0-856fc492f4a0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.253997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.254608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.254886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.255155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.255781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.257367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.268854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3878/7828] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:55.273205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.283539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e84b0-93fa-413f-a287-dc6a18037493 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.288075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.288747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.289024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.289297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] PATH_INFO: `/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8/users/d43b28e9e34c42c59801e9ed9d6b6bdc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.303654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1d2c95e9ef4bebbbda0498dfa555c8, user_id=d43b28e9e34c42c59801e9ed9d6b6bdc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.306291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.320276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3951/7829] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:55 2022] PUT /identity/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8/users/d43b28e9e34c42c59801e9ed9d6b6bdc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:55.325368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.335225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9310918-e0ce-4203-ab25-6ca9e75564f3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.392032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.392721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.392922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.393208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.393838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.395459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107487-ea96-49cd-af7d-d9693ecb82b7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.406478 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3879/7830] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:19:55.410691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.421120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a4f8e-ad97-4e3e-ba9b-d5e5f470ab16 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.425736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.426382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.426690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.426964 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] PATH_INFO: `/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8/users/d43b28e9e34c42c59801e9ed9d6b6bdc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.428869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1d2c95e9ef4bebbbda0498dfa555c8, user_id=d43b28e9e34c42c59801e9ed9d6b6bdc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.431531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.442308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3952/7831] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:19:55 2022] PUT /identity/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8/users/d43b28e9e34c42c59801e9ed9d6b6bdc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:55.453358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.453644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.453923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.482844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] MFA Rules not processed for user `d43b28e9e34c42c59801e9ed9d6b6bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:55.484101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:55.524340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:55.553503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3880/7832] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:55.565902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:55.567703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:55.629455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0aa456-ebab-4197-85b8-9533aac4e0b9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:55.633187 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:55.633791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:55.634069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:55.634342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:55.683687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:55.693456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:55.694968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12def2-d86e-438a-ac4d-7fe21b540d28 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:55.704288 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3953/7833] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:58.677836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:58.743573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23224373-31fb-4153-9911-9c60f7ed42e0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:58.749126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:58.749771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:58.750044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:58.750313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] PATH_INFO: `/v3/users/1c22c8325d7941589aad7e561caf1e71` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:58.751468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c22c8325d7941589aad7e561caf1e71)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:58.753375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:58.812311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c22c8325d7941589aad7e561caf1e71'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:58.813155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c22c8325d7941589aad7e561caf1e71'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:58.823044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c22c8325d7941589aad7e561caf1e71'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:58.831984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c22c8325d7941589aad7e561caf1e71'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:58.832669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c22c8325d7941589aad7e561caf1e71'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:58.852843 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3881/7834] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:58 2022] DELETE /identity/v3/users/1c22c8325d7941589aad7e561caf1e71 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:58.857933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:58.858402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:58.858569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:58.882827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:19:58.884046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:58.953156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3954/7835] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:19:58.962911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:58.964928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:59.028207 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bb014-a4ef-4fe7-86aa-c91387251bea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:59.031851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:59.032445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:59.032726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:59.032999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:59.043242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:59.044685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:59.048740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:19:59.058819 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3882/7836] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:19:59.380344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:59.382075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:19:59.396926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609177df-1fe1-4abc-ac7f-65b23521bf78 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:59.403863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:59.404631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:59.404977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:59.405319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] PATH_INFO: `/v3/projects/40d1d406f913492ab44d4cd5e1c97852` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:59.406623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d1d406f913492ab44d4cd5e1c97852)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:59.408731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:59.485570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d1d406f913492ab44d4cd5e1c97852', 'request_id': 'req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.492829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d1d406f913492ab44d4cd5e1c97852', 'request_id': 'req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.501266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d1d406f913492ab44d4cd5e1c97852', 'request_id': 'req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.520933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] The token cache is being invalidate because project 40d1d406f913492ab44d4cd5e1c97852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:19:59.521446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.522160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.524715 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3955/7837] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:19:59 2022] DELETE /identity/v3/projects/40d1d406f913492ab44d4cd5e1c97852 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:19:59.529553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:19:59.541568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd746d3-63df-4bee-b537-0cf67555d7f5 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:19:59.604191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:19:59.605582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:19:59.605862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:19:59.606135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] PATH_INFO: `/v3/users/d43b28e9e34c42c59801e9ed9d6b6bdc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:19:59.607298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43b28e9e34c42c59801e9ed9d6b6bdc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:19:59.609215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:19:59.661345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43b28e9e34c42c59801e9ed9d6b6bdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.662000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43b28e9e34c42c59801e9ed9d6b6bdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.671131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43b28e9e34c42c59801e9ed9d6b6bdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.681377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43b28e9e34c42c59801e9ed9d6b6bdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.682358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-0676f516-13f3-4d80-94ab-108928556522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43b28e9e34c42c59801e9ed9d6b6bdc'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:19:59.703150 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3883/7838] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:19:59 2022] DELETE /identity/v3/users/d43b28e9e34c42c59801e9ed9d6b6bdc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:20:00.002477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.003960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:00.017950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b1491-3d3f-4bfa-a8d0-b26faea64362 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.093395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.094094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.094461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.094864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] PATH_INFO: `/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.096079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1d2c95e9ef4bebbbda0498dfa555c8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.098372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.118533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:00.154263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1d2c95e9ef4bebbbda0498dfa555c8', 'request_id': 'req-4396482d-a105-4948-9930-ca340819bc1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:00.159654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1d2c95e9ef4bebbbda0498dfa555c8', 'request_id': 'req-4396482d-a105-4948-9930-ca340819bc1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:00.165716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1d2c95e9ef4bebbbda0498dfa555c8', 'request_id': 'req-4396482d-a105-4948-9930-ca340819bc1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:00.183328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] The token cache is being invalidate because project dd1d2c95e9ef4bebbbda0498dfa555c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:20:00.183837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:00.184875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4396482d-a105-4948-9930-ca340819bc1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:00.187486 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3956/7839] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:20:00 2022] DELETE /identity/v3/projects/dd1d2c95e9ef4bebbbda0498dfa555c8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:20:00.194346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.194687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.194942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.214935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:20:00.216137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:00.278346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3884/7840] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:20:00.287597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.287908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.288181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.309215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:20:00.310422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:00.318403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3957/7841] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:20:00.322610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.332594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e93d4-b56e-4d98-8a8a-290a6d1807da None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.337271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.337862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.338143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.338416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.339090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:20:00.339394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.340972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.352696 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3885/7842] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:20:00.357340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.367001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b04f7-47db-4653-bdf0-b8416551fd70 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.372434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.373025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.373330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.373605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.374234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.376195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.403073 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3958/7843] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:20:00 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:20:00.407290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.413616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:00.418450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d0d9-e507-4526-b54d-b29b8918383a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.477301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.477905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.478182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.478452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.479113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.482051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.502897 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3886/7844] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:20:00 2022] POST /identity/v3/users => generated 280 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:20:00.507084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.517659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3fd56-df77-491f-9301-997a9b7ad049 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.522285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.523180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.523458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.523728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.524353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.525924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.536154 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3959/7845] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:20:00.540008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.551251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f38e5-d065-4581-8bd8-96e0f559cc20 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.555987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.556571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.556845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.557109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.557963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.559661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfc2be-d4bf-4bce-99a1-8f70223bcb02 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.573281 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3887/7846] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:20:00.574436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.584782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36944d4d-f2c5-4fd7-8775-fe28c55775b7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.590425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.591110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.591387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.591658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] PATH_INFO: `/v3/projects/3b97fcc0a3c146d997d163db614666a2/users/4fc8b771fe524287a3c5883df8efc317/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.607351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b97fcc0a3c146d997d163db614666a2, user_id=4fc8b771fe524287a3c5883df8efc317, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.609911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.625335 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3960/7847] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:20:00 2022] PUT /identity/v3/projects/3b97fcc0a3c146d997d163db614666a2/users/4fc8b771fe524287a3c5883df8efc317/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:20:00.636409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.636835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.637662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.669403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] MFA Rules not processed for user `4fc8b771fe524287a3c5883df8efc317`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:20:00.670892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:00.739348 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3888/7848] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:20:00.772395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.774371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:00.841176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4c908-1644-456c-b1cd-dccc26957bcd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.844996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.845597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.845879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.846156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.854736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.856306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b87c48-329b-45dc-b717-f306b6504014 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.865035 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3961/7849] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:20:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:20:00.881460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:00.943600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baf507-6b3e-4493-975e-5a9205559327 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:00.949038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:00.949654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:00.949933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:00.950208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] PATH_INFO: `/v3/users/4fc8b771fe524287a3c5883df8efc317` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:00.951395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc8b771fe524287a3c5883df8efc317)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:00.953324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:00.967404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:00.999523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc8b771fe524287a3c5883df8efc317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.000170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc8b771fe524287a3c5883df8efc317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.007278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc8b771fe524287a3c5883df8efc317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.014692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc8b771fe524287a3c5883df8efc317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.015325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc8b771fe524287a3c5883df8efc317'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.033728 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3889/7850] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:20:00 2022] DELETE /identity/v3/users/4fc8b771fe524287a3c5883df8efc317 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:20:01.038274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:01.038780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:01.038880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:01.058913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:20:01.060153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:01.108230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:01.128207 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3962/7851] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:20:01.138398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:01.139815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:01.200541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ad143-9539-4cd0-8e0b-e74fb389e739 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:01.204270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:01.204872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:01.205149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:01.205419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:01.216126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:01.217569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:01.227839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3890/7852] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:20:01.514171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:01.515670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:01.530471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac97e28-4f84-4d16-aaff-8acf37e10219 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:01.535349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:01.535987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:01.536258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:01.536522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] PATH_INFO: `/v3/projects/3b97fcc0a3c146d997d163db614666a2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:01.537756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b97fcc0a3c146d997d163db614666a2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:01.539568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:01.587356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b97fcc0a3c146d997d163db614666a2', 'request_id': 'req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.593268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b97fcc0a3c146d997d163db614666a2', 'request_id': 'req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.599705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b97fcc0a3c146d997d163db614666a2', 'request_id': 'req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.613457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] The token cache is being invalidate because project 3b97fcc0a3c146d997d163db614666a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:20:01.613773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.614520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:20:01.616018 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3963/7853] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:20:01 2022] DELETE /identity/v3/projects/3b97fcc0a3c146d997d163db614666a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:20:01.751077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:01.752646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:01.769107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28593c-48b1-4c3a-9e17-a7ac68b73119 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:01.828654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:01.829270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:01.829539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:01.829806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:01.831661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:01.883902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:01.914915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:01.916295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:01.931862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3891/7854] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:20:11.610339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:11.923761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=11 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:20:48.408139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:48.409511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:48.427142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6e22-4403-4e5f-a5cb-d43dc4ab0ba5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:48.428986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:48.429636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:48.429914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:48.430182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:48.431935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:48.445141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:48.446634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:48.458032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3964/7855] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:20:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:20:55.630844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:20:55.632383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:55.649751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be13bd5-f36b-48a2-a94e-3380e9113831 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:20:55.655605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:20:55.656205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:20:55.656485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:20:55.656758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:20:55.658457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:20:55.716526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:20:55.718071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836136b-19bb-4f67-b7b0-49ab3d143f2d service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:20:55.727715 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3892/7856] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:20:58.452194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:05.723132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=28 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:18.523710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:18.525045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:18.543453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcba868-bca7-4ad0-b308-34c9a3945ea6 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:18.549021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:18.549613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:18.549892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:18.550163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:18.551860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:18.613317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:18.614974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:18.625455 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3965/7857] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:18 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:28.620237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=30 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:34.920998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:34.921967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:34.922241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:34.953158 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] MFA Rules not processed for user `82c2160fcae947138ddd7b1b4df1e361`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:21:34.954531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:34.962535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3893/7858] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:21:35.699080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:35.701080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:35.784271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a5849-f799-43b4-851b-33eaee619620 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:35.789589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:35.790196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:35.790587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:35.790761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:35.792531 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:35.808310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:35.809890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:35.823138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3966/7859] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:44.882050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:44.900161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa6b9a-4311-4f54-9226-03b0ad9b4df0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:44.906291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:44.907552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:44.907850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:44.908117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:44.909861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:44.925552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:44.927151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:44.942620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3894/7860] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:45.816148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=32 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:48.179538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:48.216963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e61e0-92db-4302-a32c-7e6b601cef2a service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:48.224708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:48.225649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:48.225858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:48.226155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:48.228745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:48.264604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:48.266230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:48.284198 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3967/7861] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:50.261220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:50.287551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fd1ad-1eec-4790-9bc6-f633cb0c8d4c service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:50.293391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:50.294737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:50.295105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:50.295382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:50.297213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:50.315127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:50.316883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:50.331379 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3895/7862] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:52.229590 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:52.252141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b0a9-0c97-43cb-80c0-1194b94f6570 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:52.255831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:52.256425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:52.256705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:52.256972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:52.258685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:52.324563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:52.326279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:52.347804 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3968/7863] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:55.571931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:55.573453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:55.602540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd35b-ed76-46db-8b96-fa5b24daf507 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:55.608389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:55.609365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:55.609862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:55.610347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:55.612520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:55.639449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:55.642038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:55.662602 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3896/7864] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:21:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:21:58.626336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:58.627923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:58.650655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce3735-6ad6-4747-a3d2-0e5ea2566bc7 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:58.656173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:58.656819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:58.657156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:58.657600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] PATH_INFO: `/v3/users/ca78cf6d34e84f2cb5cb254c5d8af1da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:58.659016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca78cf6d34e84f2cb5cb254c5d8af1da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:58.661107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:58.714057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca78cf6d34e84f2cb5cb254c5d8af1da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:58.714816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca78cf6d34e84f2cb5cb254c5d8af1da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:58.725290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca78cf6d34e84f2cb5cb254c5d8af1da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:58.738042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca78cf6d34e84f2cb5cb254c5d8af1da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:58.738742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca78cf6d34e84f2cb5cb254c5d8af1da'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:58.751092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3969/7865] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:21:58 2022] DELETE /identity/v3/users/ca78cf6d34e84f2cb5cb254c5d8af1da => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:21:58.923002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:58.928517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=30 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:58.938820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df107-34c8-4ea9-8882-4ed6e40d726a service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:58.997376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:58.997995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:58.998270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:58.998569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] PATH_INFO: `/v3/projects/f3c583f2016e4edd94aeec7dd64f9673` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:58.999673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c583f2016e4edd94aeec7dd64f9673)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.001585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.046785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c583f2016e4edd94aeec7dd64f9673', 'request_id': 'req-94d21cc2-9a99-438b-8835-550e2cee8134'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.052198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c583f2016e4edd94aeec7dd64f9673', 'request_id': 'req-94d21cc2-9a99-438b-8835-550e2cee8134'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.060130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c583f2016e4edd94aeec7dd64f9673', 'request_id': 'req-94d21cc2-9a99-438b-8835-550e2cee8134'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.071165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] The token cache is being invalidate because project f3c583f2016e4edd94aeec7dd64f9673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:21:59.071446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.072066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.073604 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3897/7866] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:21:58 2022] DELETE /identity/v3/projects/f3c583f2016e4edd94aeec7dd64f9673 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:21:59.078454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.079885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:59.095627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7acc31-5319-42bb-a0e5-99218a310946 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.119547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:59.154755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.155391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.155668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.155939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] PATH_INFO: `/v3/users/238df9aa958f4ed8944c7d4355cb40c0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.169374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238df9aa958f4ed8944c7d4355cb40c0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.171373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.216191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238df9aa958f4ed8944c7d4355cb40c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.216828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238df9aa958f4ed8944c7d4355cb40c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.223816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238df9aa958f4ed8944c7d4355cb40c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.236203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238df9aa958f4ed8944c7d4355cb40c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.236851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75f94863-e524-454c-8fd2-c89519143036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238df9aa958f4ed8944c7d4355cb40c0'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.249275 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3970/7867] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:21:59 2022] DELETE /identity/v3/users/238df9aa958f4ed8944c7d4355cb40c0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:21:59.425888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.427344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:59.443926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d21cc2-9a99-438b-8835-550e2cee8134 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.507635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.508256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.508532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.508802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] PATH_INFO: `/v3/projects/d53792b48f1248ec95d6cd73c056bb8e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.518988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53792b48f1248ec95d6cd73c056bb8e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.521181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.555338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:59.574012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53792b48f1248ec95d6cd73c056bb8e', 'request_id': 'req-5606eeb2-06f2-410e-b06a-b69508d83f3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.579684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53792b48f1248ec95d6cd73c056bb8e', 'request_id': 'req-5606eeb2-06f2-410e-b06a-b69508d83f3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.587521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53792b48f1248ec95d6cd73c056bb8e', 'request_id': 'req-5606eeb2-06f2-410e-b06a-b69508d83f3f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.600308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] The token cache is being invalidate because project d53792b48f1248ec95d6cd73c056bb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:21:59.600522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.601142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5606eeb2-06f2-410e-b06a-b69508d83f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:21:59.602663 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3898/7868] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:21:59 2022] DELETE /identity/v3/projects/d53792b48f1248ec95d6cd73c056bb8e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:21:59.609034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.609323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.609599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.632356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:21:59.634134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:59.700160 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3971/7869] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:21:59.708245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.708534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.708807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.730421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:21:59.732242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:21:59.740257 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3899/7870] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:21:59.744487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.754266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3530-b7e5-44f5-9ab8-19632ae19a92 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.758858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.759448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.759725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.759999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.760633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:21:59.760936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.762537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.774067 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3972/7871] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:21:59.778849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.788573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1e12-106b-4336-839f-008bd68442dd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.793864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.794469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.794774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.795050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.795682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.797591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.824155 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3900/7872] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:21:59 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:21:59.828492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.838703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462618a9-b7fe-401b-8b17-659b6ba8ed9c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.859519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:21:59.898769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.899365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.899642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.899910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.900533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.902957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.926648 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3973/7873] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:21:59 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:21:59.930816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.941544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17653d-9318-4114-91f8-9db0b07e53ff admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.946104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.946720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.946995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.947264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.947886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.950054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.960568 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3901/7874] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:21:59.964552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:21:59.975168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eb88c-7ed8-4502-b9bb-5265a6aebaa9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:21:59.979774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:21:59.980360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:21:59.980654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:21:59.980924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:21:59.981550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:21:59.983159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879cb67-21e5-4505-a00b-21018b366df9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:21:59.994350 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3974/7875] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:21:59.998727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.008511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453e0-fe6e-4f7d-aedf-d28c715d7b63 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.013903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.014560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.014840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.015111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26/users/8bc57c8932d642aaa8b9b9f534b628b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.030296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c066e5af8554d298fe7e8c72b349f26, user_id=8bc57c8932d642aaa8b9b9f534b628b2, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.033023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.045802 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3902/7876] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:21:59 2022] PUT /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26/users/8bc57c8932d642aaa8b9b9f534b628b2/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:00.057296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.057611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.057969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.086651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] MFA Rules not processed for user `8bc57c8932d642aaa8b9b9f534b628b2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:00.087865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:00.153876 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3975/7877] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:22:00.158489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.220790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3f04b-69c6-40b8-bece-54493ba7322f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.225483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.226077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.226353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.226656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.227269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.229198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.256059 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3903/7878] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:22:00 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:00.260155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.271838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9b3fb-b65b-445b-8a0f-9571b41ed5fe None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.327719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.328320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.328598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.328868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.329484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.331454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.353040 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3976/7879] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:22:00 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:00.357092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.367504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f1f1-a974-4fc7-8807-6510eee6328f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.372747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.373340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.373621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.373898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.374569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.376220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.386839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3904/7880] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:22:00.391557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.402064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe024c5-0b39-48be-a3ad-4488a9c81b35 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.407173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.407761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.408053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.408311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] PATH_INFO: `/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317/users/5d068d8afbe94ae4b16545019ec082ec/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.423186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ef9fce638a4ed8b0ee7dc080fb4317, user_id=5d068d8afbe94ae4b16545019ec082ec, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.425886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.429079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:00.441418 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3977/7881] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:22:00 2022] PUT /identity/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317/users/5d068d8afbe94ae4b16545019ec082ec/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:00.445361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.456723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec285cb-0806-4ab9-b7ac-52946fef50b3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.476426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:00.514261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.514890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.515175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.515449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.516085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.517682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b8050-ee73-4671-84e2-a3ff6eac967c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.527524 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3905/7882] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:22:00.531820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.542223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a93e-c28b-472a-a52d-ca756fa1519f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.547148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.547793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.548068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.548339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] PATH_INFO: `/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317/users/5d068d8afbe94ae4b16545019ec082ec/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.550241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ef9fce638a4ed8b0ee7dc080fb4317, user_id=5d068d8afbe94ae4b16545019ec082ec, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.552958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.563758 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3978/7883] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:22:00 2022] PUT /identity/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317/users/5d068d8afbe94ae4b16545019ec082ec/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:00.573938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.574963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.575382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.604350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] MFA Rules not processed for user `5d068d8afbe94ae4b16545019ec082ec`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:00.605578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:00.677497 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3906/7884] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:00.684780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.695667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0eca7-eb94-415a-b453-e56f3c7709a4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.701151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.701746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.702025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.702295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.702973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:00.703271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.704854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.717141 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3979/7885] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:00.721576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.731391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5804f6-7ca1-40d2-960d-3ddfaaf9950d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.736205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.736803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.737090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.737364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.737999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.739979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.768073 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3907/7886] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:22:00 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:22:00.772383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.782406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7befbd4-f232-4b26-80a3-ad8a94e54430 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.837908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.838517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.838821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.839118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.839737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:00.840114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.841846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db39c7-3212-4bfd-8ac6-d306550b396b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.852963 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3980/7887] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:00.864828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:00.866680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:00.942517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08bd33-6cdf-4abf-a106-274c1a539951 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:00.948026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:00.948631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.948913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.949194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.957687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:00.959341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:00.969012 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3908/7888] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:00.984924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457af7f-72a1-4082-904e-05f1da9f1b52 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:00.985212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457af7f-72a1-4082-904e-05f1da9f1b52 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:00.985487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457af7f-72a1-4082-904e-05f1da9f1b52 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:00.986842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3981/7889] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:00.991295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.005208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.010162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:01.014210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc909df-7191-448d-a96c-3ffe971f0c25 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.020125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.021047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.021326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.021595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/93f2ef7a78c348bdb36ed16846868cca` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.022747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=93f2ef7a78c348bdb36ed16846868cca)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.024813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.026415 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3909/7890] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:00 2022] GET /identity/v3/projects/93f2ef7a78c348bdb36ed16846868cca => generated 369 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.071313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6692-9bc5-498a-8f52-71bfa5fa7d11 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.071597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6692-9bc5-498a-8f52-71bfa5fa7d11 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.071869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda6692-9bc5-498a-8f52-71bfa5fa7d11 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.073179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3982/7891] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:01.078267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.080002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:01.099555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.108822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b74d-5c21-4a24-8837-60db36d101e1 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.113411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.114016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.114291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.114587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/93f2ef7a78c348bdb36ed16846868cca` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.115673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=93f2ef7a78c348bdb36ed16846868cca)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.117713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.119326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3910/7892] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/projects/93f2ef7a78c348bdb36ed16846868cca => generated 369 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.225600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21dc8-6b50-4d38-9aad-46fdb6b3dfcd None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.225881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21dc8-6b50-4d38-9aad-46fdb6b3dfcd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.226157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a21dc8-6b50-4d38-9aad-46fdb6b3dfcd None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.227627 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3983/7893] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:01.233307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.248626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.257550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7c1e0-3de2-496a-8b8d-499024235402 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.263109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.263719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.263996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.264264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/93f2ef7a78c348bdb36ed16846868cca` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.265373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=93f2ef7a78c348bdb36ed16846868cca)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.267478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978d5a-94f5-4a3e-ba28-b102c45f8488 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.269094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3911/7894] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/projects/93f2ef7a78c348bdb36ed16846868cca => generated 369 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.294359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21dc8-6b50-4d38-9aad-46fdb6b3dfcd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.306724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a21dc8-6b50-4d38-9aad-46fdb6b3dfcd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.311618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.312228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.312506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.312779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/93f2ef7a78c348bdb36ed16846868cca` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.313893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:delete_project(project_id=93f2ef7a78c348bdb36ed16846868cca)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.315807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.362105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f2ef7a78c348bdb36ed16846868cca', 'request_id': 'req-04c227a3-1218-4e6d-8941-ce076242b21c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:01.367553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f2ef7a78c348bdb36ed16846868cca', 'request_id': 'req-04c227a3-1218-4e6d-8941-ce076242b21c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:01.374249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f2ef7a78c348bdb36ed16846868cca', 'request_id': 'req-04c227a3-1218-4e6d-8941-ce076242b21c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:01.387602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] The token cache is being invalidate because project 93f2ef7a78c348bdb36ed16846868cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:22:01.387895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:01.388556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:01.390054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3984/7895] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:22:01 2022] DELETE /identity/v3/projects/93f2ef7a78c348bdb36ed16846868cca => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:01.407829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022cce-1967-4ee4-be6a-c01e87b1c3e8 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.408109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022cce-1967-4ee4-be6a-c01e87b1c3e8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.408379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32022cce-1967-4ee4-be6a-c01e87b1c3e8 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.409692 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3912/7896] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:01.414328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.415719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:01.465668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:01.494699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.495921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:01.559176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c227a3-1218-4e6d-8941-ce076242b21c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.563839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.564473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.564756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.565035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.566166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=6c066e5af8554d298fe7e8c72b349f26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.568274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.569948 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3985/7897] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26 => generated 373 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.591022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022cce-1967-4ee4-be6a-c01e87b1c3e8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.603487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32022cce-1967-4ee4-be6a-c01e87b1c3e8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.608046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.608632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.608913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.609188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.609829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:01.610138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.611765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.623805 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3913/7898] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.627661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.640129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6da4d-52b0-4035-b2bb-cef515e90ab9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.645361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.646225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.646539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.646817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.647464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.649361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c04ed-da30-43b5-a3cb-815c01685ebb tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.676825 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3986/7899] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:22:01 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:01.680932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.691268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd44e5-f56c-4c48-b9ea-b90a66fc6780 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.747792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.748343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.748624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.748898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.749537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:01.749836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.751471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.763489 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3914/7900] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:01.782681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b08-1ebb-449f-aad7-284bb81ebc13 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.783173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b08-1ebb-449f-aad7-284bb81ebc13 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.783271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b08-1ebb-449f-aad7-284bb81ebc13 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.784710 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3987/7901] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:01.789464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.791177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:01.866134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:01.875294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0614f-6a92-430e-83b8-e00f6d5e0af3 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:01.879908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:01.880516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.880802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.881074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/7742ae37c4424c67a9b1b59d962e8f05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.882194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=7742ae37c4424c67a9b1b59d962e8f05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:01.884285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:01.885914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3915/7902] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/projects/7742ae37c4424c67a9b1b59d962e8f05 => generated 367 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:22:01.971983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa24786-2c24-489b-bb27-b4f6d5914fa4 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:01.972287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa24786-2c24-489b-bb27-b4f6d5914fa4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:01.972546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa24786-2c24-489b-bb27-b4f6d5914fa4 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:01.973888 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3988/7903] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:01.978522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:01.993694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.003263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a900b-9f5f-4b91-bc79-186982c3098d tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.007813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.008415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.008692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.008960 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/7742ae37c4424c67a9b1b59d962e8f05` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.010066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=7742ae37c4424c67a9b1b59d962e8f05)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.012877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.014813 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3916/7904] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:01 2022] GET /identity/v3/projects/7742ae37c4424c67a9b1b59d962e8f05 => generated 367 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:02.062552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa24786-2c24-489b-bb27-b4f6d5914fa4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.076076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa24786-2c24-489b-bb27-b4f6d5914fa4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.080537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.081124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.081404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.081672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.082294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:02.082631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.084206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.096373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3989/7905] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:02.100498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.110446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a88c56-2b0c-432d-a5ca-4055186f4fca tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.115065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.115642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.115920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.116190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.116809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.118727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9c56c0-29a4-4711-9222-bfa921e929c7 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.141574 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3917/7906] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:22:02 2022] POST /identity/v3/users => generated 383 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:02.145771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.155792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20ec90-0021-4684-ba0e-3ffcfc2be41c tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.160800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.161393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.161674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.161971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.162642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:02.162942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.164539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.176152 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3990/7907] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:02.195335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e68f-76f9-45e4-b88c-3fef8916b085 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.195622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e68f-76f9-45e4-b88c-3fef8916b085 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.195901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e68f-76f9-45e4-b88c-3fef8916b085 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.197231 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3918/7908] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:02.202485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:02.218144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.227202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3187d2-ac8a-48f3-9f46-ba43301bb64b tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.232545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.233167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.233449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.233719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/7742ae37c4424c67a9b1b59d962e8f05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.234875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=7742ae37c4424c67a9b1b59d962e8f05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.236900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.238527 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3991/7909] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/projects/7742ae37c4424c67a9b1b59d962e8f05 => generated 367 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:02.364362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8386b2-6071-420c-97f7-bb345f5ce394 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.364652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8386b2-6071-420c-97f7-bb345f5ce394 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.364915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8386b2-6071-420c-97f7-bb345f5ce394 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.366232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3919/7910] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:02.370926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:02.386746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.396449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e67a8a-73b7-4bd4-835e-875c28a89b04 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.401233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.401850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.402128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.402410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/7742ae37c4424c67a9b1b59d962e8f05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.403557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=7742ae37c4424c67a9b1b59d962e8f05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.405560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.407378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3992/7911] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/projects/7742ae37c4424c67a9b1b59d962e8f05 => generated 367 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:02.440187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8386b2-6071-420c-97f7-bb345f5ce394 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.452908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8386b2-6071-420c-97f7-bb345f5ce394 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.457867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.458467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.458772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.459047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/users/b58197dcdf644ab7b9bd5702f56b9718` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.473097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:delete_user(user_id=b58197dcdf644ab7b9bd5702f56b9718)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.475108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.485625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:02.519404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58197dcdf644ab7b9bd5702f56b9718'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.520062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58197dcdf644ab7b9bd5702f56b9718'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.527857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58197dcdf644ab7b9bd5702f56b9718'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.535002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58197dcdf644ab7b9bd5702f56b9718'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.535723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a62f5d01-6d5b-4f4a-8d36-f78da5c54210 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58197dcdf644ab7b9bd5702f56b9718'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.553902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3920/7912] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:22:02 2022] DELETE /identity/v3/users/b58197dcdf644ab7b9bd5702f56b9718 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:02.558088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.560664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:02.574191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62854d-ea31-4270-a01b-7c4ca1d333b9 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.631983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.632615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.632890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.633157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/7742ae37c4424c67a9b1b59d962e8f05` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.638153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7742ae37c4424c67a9b1b59d962e8f05)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.638153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.658987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:02.685166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7742ae37c4424c67a9b1b59d962e8f05', 'request_id': 'req-ca2ca674-5a38-4cae-896b-4789d2c065dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.690571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7742ae37c4424c67a9b1b59d962e8f05', 'request_id': 'req-ca2ca674-5a38-4cae-896b-4789d2c065dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.696524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7742ae37c4424c67a9b1b59d962e8f05', 'request_id': 'req-ca2ca674-5a38-4cae-896b-4789d2c065dc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.709039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] The token cache is being invalidate because project 7742ae37c4424c67a9b1b59d962e8f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:22:02.709327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.709963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:02.711571 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3993/7913] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:22:02 2022] DELETE /identity/v3/projects/7742ae37c4424c67a9b1b59d962e8f05 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:02.728700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedba77-d5c6-4e11-b750-a4697a16830c None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.729013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedba77-d5c6-4e11-b750-a4697a16830c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.729294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedba77-d5c6-4e11-b750-a4697a16830c None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.731395 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3921/7914] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:02.736062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:02.737451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:02.813925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.815135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:02.877713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca674-5a38-4cae-896b-4789d2c065dc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.882378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.883035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.883313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.883584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.884721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=6c066e5af8554d298fe7e8c72b349f26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.886793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.888427 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3994/7915] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26 => generated 373 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:22:02.921671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aed39-d084-4f05-a914-883f2c66143e None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.921953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aed39-d084-4f05-a914-883f2c66143e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.922224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aed39-d084-4f05-a914-883f2c66143e None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.923651 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3922/7916] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:02.928057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:02.929749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:02.952281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:02.967953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2f7cd-1808-42f4-a008-24aa7b1f3041 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:02.977128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:02.978196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:02.978720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:02.979170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:02.980460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=6c066e5af8554d298fe7e8c72b349f26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:02.983114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:02.984816 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3995/7917] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:02 2022] GET /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26 => generated 373 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:22:03.225465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:03.226010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:03.226262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:03.227640 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3923/7918] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:22:03.232547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:22:03.248352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:03.257584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705a8e7-5178-4e10-8853-95adf36fa719 tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:03.262922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d068d8afbe94ae4b16545019ec082ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:03.263546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:03.263823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:03.264087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:03.265237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorizing `identity:get_project(project_id=6c066e5af8554d298fe7e8c72b349f26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:03.267322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffd9e6-2bed-46fd-8902-7b067b33fafc tempest-QuotasAdminTestJSON-113582718 tempest-QuotasAdminTestJSON-113582718-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:03.269102 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3996/7919] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:22:03 2022] GET /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26 => generated 373 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:03.610117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:03.612630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:03.692040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295f5db-6908-451c-b66d-38b8431f1f67 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:03.697362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:03.697986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:03.698262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:03.698559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] PATH_INFO: `/v3/users/8bc57c8932d642aaa8b9b9f534b628b2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:03.700559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc57c8932d642aaa8b9b9f534b628b2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:03.703743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:03.758162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc57c8932d642aaa8b9b9f534b628b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:03.758857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc57c8932d642aaa8b9b9f534b628b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:03.767565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc57c8932d642aaa8b9b9f534b628b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:03.775284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc57c8932d642aaa8b9b9f534b628b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:03.775945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc57c8932d642aaa8b9b9f534b628b2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:03.793949 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3924/7920] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:22:03 2022] DELETE /identity/v3/users/8bc57c8932d642aaa8b9b9f534b628b2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:03.798387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:03.798694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:03.798970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:03.818219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:03.819535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:03.839284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:03.881735 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3997/7921] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:03.891401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:03.892856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:03.903247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:03.955445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb375b-603c-4a5d-a3f5-f217986e6f31 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:03.959161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:03.959761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:03.960067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:03.960343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:03.969537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:03.971017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:03.981237 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3925/7922] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:04.255686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:04.257141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:04.276819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79113565-73e2-413f-86fc-e44fa78a3461 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:04.285918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:04.286609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:04.286896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:04.287169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] PATH_INFO: `/v3/projects/6c066e5af8554d298fe7e8c72b349f26` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:04.288347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c066e5af8554d298fe7e8c72b349f26)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:04.290294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:04.338302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c066e5af8554d298fe7e8c72b349f26', 'request_id': 'req-61ecbabf-8549-4832-8f31-e58d5ab2360a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.344469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c066e5af8554d298fe7e8c72b349f26', 'request_id': 'req-61ecbabf-8549-4832-8f31-e58d5ab2360a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.350438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c066e5af8554d298fe7e8c72b349f26', 'request_id': 'req-61ecbabf-8549-4832-8f31-e58d5ab2360a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.363087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] The token cache is being invalidate because project 6c066e5af8554d298fe7e8c72b349f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:22:04.363387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.364016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.365530 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3998/7923] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:22:04 2022] DELETE /identity/v3/projects/6c066e5af8554d298fe7e8c72b349f26 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:22:04.369728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:04.379910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc54d2b-4e2c-4e00-8b50-03672834d2ab service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:04.436406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:04.436947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:04.437226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:04.437499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] PATH_INFO: `/v3/users/5d068d8afbe94ae4b16545019ec082ec` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:04.438654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d068d8afbe94ae4b16545019ec082ec)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:04.440573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:04.485991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d068d8afbe94ae4b16545019ec082ec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.486670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d068d8afbe94ae4b16545019ec082ec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.493720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d068d8afbe94ae4b16545019ec082ec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.500793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d068d8afbe94ae4b16545019ec082ec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.501442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-72829430-250b-4982-a8be-2edfefbb4881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d068d8afbe94ae4b16545019ec082ec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.519482 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3926/7924] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:22:04 2022] DELETE /identity/v3/users/5d068d8afbe94ae4b16545019ec082ec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:04.800743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:04.802219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:04.815639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecbabf-8549-4832-8f31-e58d5ab2360a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:04.872561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:04.873218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:04.873464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:04.873737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] PATH_INFO: `/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:04.874878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ef9fce638a4ed8b0ee7dc080fb4317)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:04.876812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:04.922290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'request_id': 'req-8bfca77e-9567-4485-9e76-7c653f1f9590'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.924212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:04.928676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'request_id': 'req-8bfca77e-9567-4485-9e76-7c653f1f9590'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.936785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ef9fce638a4ed8b0ee7dc080fb4317', 'request_id': 'req-8bfca77e-9567-4485-9e76-7c653f1f9590'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.948685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] The token cache is being invalidate because project b7ef9fce638a4ed8b0ee7dc080fb4317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:22:04.948994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.950163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8bfca77e-9567-4485-9e76-7c653f1f9590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:22:04.951672 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 3999/7925] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:22:04 2022] DELETE /identity/v3/projects/b7ef9fce638a4ed8b0ee7dc080fb4317 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:04.957918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:04.958225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:04.958526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:04.979958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:04.981107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:05.004605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:05.041175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3927/7926] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:05.049714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.049995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.050267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.071147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:05.072341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:05.080942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4000/7927] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:05.085131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.094930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa5bb7-5067-4f96-98d5-57dc7dea6330 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.099478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.100066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.100343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.100615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.101254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:22:05.101553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.103242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.114788 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3928/7928] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:22:05.118688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.128815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972f774-4128-4f72-bc5e-39467a5114f5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.133589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.134177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.134453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.134752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.135368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.138191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.173867 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4001/7929] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:22:05 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:05.177938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.188047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50392680-fdcf-428e-810a-b688ab18837f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.243401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.243922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.244202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.244477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.245105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.247112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.268232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3929/7930] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:22:05 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:05.272583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.283365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b43eee-6c94-4bbf-b83c-7a96a55f2e76 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.288028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.288662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.288930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.289201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.289830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.291461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.301204 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4002/7931] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:22:05.305019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.315975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ceffec-f1b4-43e7-92cd-2bc91c36fca6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.320680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.321278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.321555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.321826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.322489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.324712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9066e-ce8c-4a91-ad5b-97f0ef8c61e0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.337895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3930/7932] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:22:05.342838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.353547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616767ff-7658-49d8-b39f-c562ff9f7f0f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.357999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.358687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.358970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.359251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] PATH_INFO: `/v3/projects/3481b55b9dfd4a25909a3fe6a03f06e3/users/0ea6653831a54a869daaba5e51fd04b8/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.374741 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3481b55b9dfd4a25909a3fe6a03f06e3, user_id=0ea6653831a54a869daaba5e51fd04b8, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.377414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.391057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4003/7933] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:22:05 2022] PUT /identity/v3/projects/3481b55b9dfd4a25909a3fe6a03f06e3/users/0ea6653831a54a869daaba5e51fd04b8/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:22:05.396376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4478fe-7510-4233-b4a7-a490935ed3dd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.396848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4478fe-7510-4233-b4a7-a490935ed3dd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.396966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4478fe-7510-4233-b4a7-a490935ed3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.416920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-be4478fe-7510-4233-b4a7-a490935ed3dd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:05.418145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-be4478fe-7510-4233-b4a7-a490935ed3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:05.481475 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3931/7934] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:05.491546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:05.496004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:05.557589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c5cc5-f9ae-4fa6-8eac-a95a7493d848 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:05.562104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:05.562744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:05.563019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:05.563286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:05.572579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:05.574030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:05.584071 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4004/7935] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:10.025887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.026380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.026666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.065211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] MFA Rules not processed for user `0ea6653831a54a869daaba5e51fd04b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:10.066434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:10.095602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:10.140183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3932/7936] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:22:10.145231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.156350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bb4d5-1945-4e7d-a967-96214ff21c79 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.161152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.161750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.162027 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.162294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.163001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.164896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.194529 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4005/7937] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:22:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:22:10.197102 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.209903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7c3c-c54f-4e2b-bd01-9cc20769abcd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.273093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.273699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.273980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.274252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.274937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.277444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.307288 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3933/7938] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:22:10 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:22:10.308739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.320352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278f83f-9864-4ff3-8e72-2565bab84ebc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.325897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.326529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.326819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.327086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.327739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.329373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.341903 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4006/7939] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:22:10.346017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.357565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7d18c-cd6f-4edf-a660-26edbcff572f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.362323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.363070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.363366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.363621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] PATH_INFO: `/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7/users/5c05d8e4c784494884cc3e662e1d59a3/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.381465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c849f089bd54f29a8fc4b717bd05ef7, user_id=5c05d8e4c784494884cc3e662e1d59a3, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.384196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.399447 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3934/7940] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:22:10 2022] PUT /identity/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7/users/5c05d8e4c784494884cc3e662e1d59a3/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:22:10.403684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.414002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9d48-12dd-4087-a948-a16511e5e928 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.461142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:10.477189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.477796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.478075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.478344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.479008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.481686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbe347-d549-48ef-85fb-da5a3fb0a743 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.492239 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4007/7941] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:22:10.497095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:10.509578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27931e-4c57-4c61-9c62-1372d7400bfd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:10.515123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:10.515868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:10.516232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:10.516573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] PATH_INFO: `/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7/users/5c05d8e4c784494884cc3e662e1d59a3/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:10.518770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c849f089bd54f29a8fc4b717bd05ef7, user_id=5c05d8e4c784494884cc3e662e1d59a3, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:10.521917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:10.533890 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3935/7942] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:22:10 2022] PUT /identity/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7/users/5c05d8e4c784494884cc3e662e1d59a3/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:22:14.819981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9eb49-6aeb-4a9b-9d06-e59a9064e15a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:14.820447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9eb49-6aeb-4a9b-9d06-e59a9064e15a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:14.820644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9eb49-6aeb-4a9b-9d06-e59a9064e15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:14.868253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ffa9eb49-6aeb-4a9b-9d06-e59a9064e15a None None] MFA Rules not processed for user `5c05d8e4c784494884cc3e662e1d59a3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:14.869696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ffa9eb49-6aeb-4a9b-9d06-e59a9064e15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:14.986596 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4008/7943] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:14.995631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:14.997630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:15.079754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3760f-e020-4b02-b4b8-2599334b09d0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:15.083867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:15.084487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:15.084804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:15.085074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:15.095823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:15.097290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:15.107770 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3936/7944] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:24.978661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=42 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:25.103002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:26.166568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:26.167112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:26.167210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:26.219945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:26.221196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:26.285895 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4009/7945] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:22:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:26.339877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:26.341329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:26.436766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ba9d9-deca-4782-9c1b-cde766bc3639 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:26.442304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:26.443054 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:26.443337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:26.443611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:26.452147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:26.453755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da888d2-f785-404c-85f5-63c412eb9e8b service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:26.461902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3937/7946] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:22:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:22:35.182060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:35.184561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:35.259488 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14fb64-d457-4866-bbce-5849b3d72e0b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:35.265149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:35.265751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:35.266033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:35.266313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:35.327365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:35.328886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:35.337775 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4010/7947] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:36.458817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=41 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:36.596239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22e95-669f-423f-82d0-caf4fce340ca None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:36.596526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22e95-669f-423f-82d0-caf4fce340ca None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:36.596801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f22e95-669f-423f-82d0-caf4fce340ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:36.616088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c3f22e95-669f-423f-82d0-caf4fce340ca None None] MFA Rules not processed for user `0ea6653831a54a869daaba5e51fd04b8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:36.617286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c3f22e95-669f-423f-82d0-caf4fce340ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:36.625136 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3938/7948] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:36.940601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:36.950275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:36.998181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8866-dc8f-4d7d-8266-a8915c40c722 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:37.001848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:37.002395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:37.002707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:37.002982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:37.004750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:37.016625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:37.018048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:37.029248 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4011/7949] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:38.695732 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:38.696247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:38.696357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:38.716290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:22:38.717620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:38.724963 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3939/7950] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:22:38.737566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:38.752370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eb03a-6b40-47f4-9418-11edf8916b66 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:38.757433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:38.758034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:38.758314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:38.758622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:38.767096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:38.768725 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:38.777419 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4012/7951] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:22:46.433560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:46.435610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:46.450607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39b5b7-fcbc-4af1-bd30-af858114a616 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:46.454483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:46.455141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:46.455495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:46.455677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:46.457487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:46.470349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:46.471793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:46.482345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3940/7952] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:48.774055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=28 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:22:53.173476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:53.174958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:53.256745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e061e7a-f8a2-4e99-970f-72a965b7ef4e service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:53.262885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:53.263512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:53.263848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:53.264059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:53.265880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:53.281598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:53.283271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:53.297013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4013/7953] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:53.481022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:53.495782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d55574-0b4b-461d-abc6-f5036b1374e7 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:53.500252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:53.500876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:53.501328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:53.501601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:53.503351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:53.517716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:53.519368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:53.532029 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3941/7954] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:53.618061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:53.631137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9e6ea-2370-429c-8e75-7839aba3a145 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:53.635130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:53.635735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:53.636013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:53.636282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:53.638790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:53.656535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:53.657970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:53.671543 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4014/7955] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:22:57.669406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:22:57.684813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b019fb-ce01-4947-b1aa-5efea898d63a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:22:57.689969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:22:57.690609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:22:57.690889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:22:57.691160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:22:57.693007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:22:57.707770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:22:57.709357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:22:57.722055 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3942/7956] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:22:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:03.664111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=38 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:05.398165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:05.412472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d91233-b42a-4f84-8010-4e66527ef1be service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:05.418144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:05.419183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:05.419183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:05.419538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:05.421553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:05.436898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:05.438554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0728ebe-4c6a-4419-b1ed-cff5384e417c service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:05.451912 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4015/7957] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:07.715690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=24 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:11.598908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:11.613576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b5cb-c542-4cd4-9ea6-f61555a5c41d service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:11.619301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:11.620640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:11.621035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:11.621442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:11.623387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:11.657115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:11.659420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938207-75b3-4395-b838-2dfc30d6e945 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:11.676078 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3943/7958] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:15.444548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:17.495833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:17.496885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:17.497178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:17.517704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] MFA Rules not processed for user `82c2160fcae947138ddd7b1b4df1e361`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:17.518980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:17.585026 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4016/7959] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:21.666812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:27.578162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=25 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:36.398126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:36.398624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:36.398757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:36.418056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:36.420082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:36.427464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3944/7960] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:23:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:36.448833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:36.463562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755bfcb-377d-4b11-8d28-076ec1eeb47d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:36.469440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:36.470091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:36.470619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:36.470896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:36.479965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:36.482285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:36.491338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4017/7961] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:23:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:23:43.309459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:43.372558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da456b4-c9aa-4957-a733-64b2a171934a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:43.377140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:43.377761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:43.378040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:43.378314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] PATH_INFO: `/v3/users/0ea6653831a54a869daaba5e51fd04b8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:43.379511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea6653831a54a869daaba5e51fd04b8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:43.381435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:43.426437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea6653831a54a869daaba5e51fd04b8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.427153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea6653831a54a869daaba5e51fd04b8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.434858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea6653831a54a869daaba5e51fd04b8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.441712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea6653831a54a869daaba5e51fd04b8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.442332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea6653831a54a869daaba5e51fd04b8'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.459065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3945/7962] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:23:43 2022] DELETE /identity/v3/users/0ea6653831a54a869daaba5e51fd04b8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:43.643278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:43.662184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11a36-41fb-4158-bd44-02cb9b21da14 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:43.720434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:43.721066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:43.721345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:43.721617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] PATH_INFO: `/v3/projects/3481b55b9dfd4a25909a3fe6a03f06e3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:43.722774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3481b55b9dfd4a25909a3fe6a03f06e3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:43.724688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:43.771720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3481b55b9dfd4a25909a3fe6a03f06e3', 'request_id': 'req-61a32eeb-0b18-433d-835c-33aaa4681c00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.777167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3481b55b9dfd4a25909a3fe6a03f06e3', 'request_id': 'req-61a32eeb-0b18-433d-835c-33aaa4681c00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.784059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3481b55b9dfd4a25909a3fe6a03f06e3', 'request_id': 'req-61a32eeb-0b18-433d-835c-33aaa4681c00'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.797239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] The token cache is being invalidate because project 3481b55b9dfd4a25909a3fe6a03f06e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:23:43.797528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.798145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.799702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4018/7963] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:23:43 2022] DELETE /identity/v3/projects/3481b55b9dfd4a25909a3fe6a03f06e3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:43.803665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:43.805052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:43.818407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd24b-e3c6-474d-97ff-2915e65a0214 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:43.874761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:43.875388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:43.875668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:43.875943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] PATH_INFO: `/v3/users/5c05d8e4c784494884cc3e662e1d59a3` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:43.877115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c05d8e4c784494884cc3e662e1d59a3)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:43.879106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:43.892352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:43.926991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c05d8e4c784494884cc3e662e1d59a3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.927672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c05d8e4c784494884cc3e662e1d59a3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.935212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c05d8e4c784494884cc3e662e1d59a3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.942181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c05d8e4c784494884cc3e662e1d59a3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.942827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-14698459-863c-420b-88a5-e9142181cc61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c05d8e4c784494884cc3e662e1d59a3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:43.959708 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3946/7964] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:23:43 2022] DELETE /identity/v3/users/5c05d8e4c784494884cc3e662e1d59a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:44.132081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.133629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:44.148074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a32eeb-0b18-433d-835c-33aaa4681c00 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.209698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.210335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.210648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.210920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] PATH_INFO: `/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.212034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c849f089bd54f29a8fc4b717bd05ef7)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.213938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.235290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:44.267008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c849f089bd54f29a8fc4b717bd05ef7', 'request_id': 'req-0b26dde1-5884-4f08-a35a-2543fd3ed644'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:44.272838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c849f089bd54f29a8fc4b717bd05ef7', 'request_id': 'req-0b26dde1-5884-4f08-a35a-2543fd3ed644'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:44.279636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c849f089bd54f29a8fc4b717bd05ef7', 'request_id': 'req-0b26dde1-5884-4f08-a35a-2543fd3ed644'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:44.294066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] The token cache is being invalidate because project 3c849f089bd54f29a8fc4b717bd05ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:23:44.294365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:44.295334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0b26dde1-5884-4f08-a35a-2543fd3ed644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:44.296962 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4019/7965] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:23:44 2022] DELETE /identity/v3/projects/3c849f089bd54f29a8fc4b717bd05ef7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:44.303343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.304166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.304724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.328989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:44.330204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:44.390369 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3947/7966] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:44.398516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.398879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.399151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.419521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:44.420737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:44.428588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4020/7967] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:44.432878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.443508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb6a89-4016-49ff-9d66-a7237a2e9b02 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.448122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.448711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.448988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.449841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.450602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:23:44.450908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.452527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.463825 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3948/7968] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:23:44.468556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.477889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124191-d3ef-4453-9ef6-bc8e3de769ce None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.482643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.483235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.483514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.483783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.484398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.486316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.514246 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4021/7969] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:23:44 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:23:44.518577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.529100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9027a-c339-4980-8ac5-f35021fb95e5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.584553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.585153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.585431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.585701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.586346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.588309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.608877 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3949/7970] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:23:44 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:23:44.613088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.624217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30614494-2efa-4c57-af10-3d0544ad540e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.629144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.629509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.629824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.630103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.630781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.632376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.643333 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4022/7971] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:23:44.647993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.659037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792303bb-e537-4fae-90aa-a0a3369b44cc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.663769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.664359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.664638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.664938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.665567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.667221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee32a0-99d6-4437-a7e3-38ffec0b851d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.669332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:44.679963 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3950/7972] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:23:44.684356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.694673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db5b5b-973f-4dfa-8710-878b8703cd3b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.700028 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.700784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.701180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.701380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] PATH_INFO: `/v3/projects/78ab797f6e43434e8560af61d4311d96/users/c48d18d96d0f4345b72d044f7bf49ef9/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.716031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ab797f6e43434e8560af61d4311d96, user_id=c48d18d96d0f4345b72d044f7bf49ef9, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.718708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.732318 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4023/7973] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:23:44 2022] PUT /identity/v3/projects/78ab797f6e43434e8560af61d4311d96/users/c48d18d96d0f4345b72d044f7bf49ef9/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:44.737401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01048e0f-6a66-4095-9087-cd35d2c2b3d1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.737684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01048e0f-6a66-4095-9087-cd35d2c2b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.737983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01048e0f-6a66-4095-9087-cd35d2c2b3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.758751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-01048e0f-6a66-4095-9087-cd35d2c2b3d1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:44.759946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-01048e0f-6a66-4095-9087-cd35d2c2b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:44.819374 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3951/7974] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:44.829365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:44.831130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:44.890658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6b3b9-39be-46bf-a632-633f4c508fb0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:44.894550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:44.895261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:44.895546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:44.895819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:44.904926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:44.906362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:44.915771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4024/7975] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:49.070323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:49.070940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:49.071055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:49.101317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] MFA Rules not processed for user `c48d18d96d0f4345b72d044f7bf49ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:49.103405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:49.175171 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3952/7976] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:49.189430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:49.191509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:49.269610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:49.306163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54308d-3419-4a32-80bf-0e3b8fd1beca service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:49.312784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:49.313401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:49.313680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:49.313951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:49.322632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:49.324547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:49.334586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4025/7977] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:49.393362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:49.395112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:49.472543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0e94d-e6c1-428b-8613-f984346b1a55 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:49.476519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:49.477230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:49.477510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:49.477783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:49.479617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] Could not find project: f3c583f2016e4edd94aeec7dd64f9673.: keystone.exception.ProjectNotFound: Could not find project: f3c583f2016e4edd94aeec7dd64f9673. Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 24 21:23:49.504224 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:23:49.505491 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:23:49.506977 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:23:49.508697 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:23:49.509984 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f3c583f2016e4edd94aeec7dd64f9673. Sep 24 21:23:49.509984 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:23:49.509984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3953/7978] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:49 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 113 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:23:52.656476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:52.674894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f9265-0635-46c9-8ccb-236ed7ef0fbd service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:52.685388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:52.686218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:52.686727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:52.687072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] PATH_INFO: `/v3/users/c48d18d96d0f4345b72d044f7bf49ef9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:52.688551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48d18d96d0f4345b72d044f7bf49ef9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:52.690956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:52.760467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48d18d96d0f4345b72d044f7bf49ef9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:52.761943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48d18d96d0f4345b72d044f7bf49ef9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:52.779869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48d18d96d0f4345b72d044f7bf49ef9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:52.802453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48d18d96d0f4345b72d044f7bf49ef9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:52.803230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5c7a1c6-ece1-4255-bced-819fc24b469d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48d18d96d0f4345b72d044f7bf49ef9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:52.818687 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4026/7979] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:23:52 2022] DELETE /identity/v3/users/c48d18d96d0f4345b72d044f7bf49ef9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:53.042024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.062525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da7ee9-9045-450e-944e-5802ae11d132 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.095888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:53.132357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.132992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.133278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.133556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] PATH_INFO: `/v3/projects/78ab797f6e43434e8560af61d4311d96` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.134756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ab797f6e43434e8560af61d4311d96)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.136995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.188351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ab797f6e43434e8560af61d4311d96', 'request_id': 'req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:53.193947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ab797f6e43434e8560af61d4311d96', 'request_id': 'req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:53.201234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ab797f6e43434e8560af61d4311d96', 'request_id': 'req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:53.216015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] The token cache is being invalidate because project 78ab797f6e43434e8560af61d4311d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:23:53.216406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:53.217390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-67b1ebe8-9a2c-4ac5-8516-98d249ef4141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:23:53.219105 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3954/7980] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:23:53 2022] DELETE /identity/v3/projects/78ab797f6e43434e8560af61d4311d96 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:53.223944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.224226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.224502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.244376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:53.245568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:53.304554 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4027/7981] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:53.313303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.313588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.313861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.333567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:53.334779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:53.342745 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3955/7982] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:53.346918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.357454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc063ab-d4b2-4c4d-bb73-ea919d6a5a64 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.362272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.362955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.363239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.363515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.364160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:23:53.364467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.366062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.378423 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4028/7983] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:23:53.382973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.392382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8062e-8ecc-4b02-8916-693baa4a2a26 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.397387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.397971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.398260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.398563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.399217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.401115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.428005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3956/7984] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:23:53 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:23:53.432129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.442915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103e4a-1312-40ef-8cfc-7c3f49f2ac3f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.471749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:53.502999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.503597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.503867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.504134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.504736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.506661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.528800 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4029/7985] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:23:53 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:23:53.532911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.543825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a7abb-0f8f-4e94-a3c7-06d06dfa9298 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.548454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.549044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.549322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.549594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.550221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.551881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.561603 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3957/7986] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:23:53.565685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.575919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8bde-e58f-4f39-80aa-a4fc041bf481 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.580615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.581240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.581519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.581792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.582432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.584060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30503df0-a32a-4abb-9672-568072e21e95 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.594551 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4030/7987] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:23:53.598707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.609612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c9d3b-7903-4a05-abc0-2c122293e46b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.614394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.615110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.615390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.615661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] PATH_INFO: `/v3/projects/ca4a40ad2f664bd9bea4b640d6ab8fa1/users/1001f73ddb3c459c88073d9342b96ec3/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.630287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4a40ad2f664bd9bea4b640d6ab8fa1, user_id=1001f73ddb3c459c88073d9342b96ec3, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.633029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.648980 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3958/7988] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:23:53 2022] PUT /identity/v3/projects/ca4a40ad2f664bd9bea4b640d6ab8fa1/users/1001f73ddb3c459c88073d9342b96ec3/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:23:53.654769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443698-e59c-44a2-b1eb-0cd6411fd668 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.654945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443698-e59c-44a2-b1eb-0cd6411fd668 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.655229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443698-e59c-44a2-b1eb-0cd6411fd668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.674814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-55443698-e59c-44a2-b1eb-0cd6411fd668 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:53.676011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55443698-e59c-44a2-b1eb-0cd6411fd668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:53.733765 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4031/7989] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:53.743685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:53.745226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:53.812323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3042c-4ccd-4b54-939f-630925cddb30 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:53.816105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:53.816703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:53.817004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:53.817273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:53.825066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:23:53.827713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:53.829266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:53.841104 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3959/7990] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:23:58.192376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:58.192930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:58.193202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:58.232509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] MFA Rules not processed for user `1001f73ddb3c459c88073d9342b96ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:23:58.233875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:58.326207 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4032/7991] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:23:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:23:58.341147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:23:58.343534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:23:58.451244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ecd9e-778d-4b0c-9026-197780c2d8b6 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:23:58.457310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:23:58.458338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:23:58.458338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:23:58.458774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:23:58.468998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:23:58.470546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:23:58.481697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3960/7992] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:23:58 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:24:01.623356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:01.663940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840127e-9afb-4631-8a10-be89a6d9254b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:01.669040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:01.669664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:01.669962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:01.670234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] PATH_INFO: `/v3/users/1001f73ddb3c459c88073d9342b96ec3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:01.671397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1001f73ddb3c459c88073d9342b96ec3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:01.673300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:01.685167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:01.721532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1001f73ddb3c459c88073d9342b96ec3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:01.722176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1001f73ddb3c459c88073d9342b96ec3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:01.728998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1001f73ddb3c459c88073d9342b96ec3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:01.741854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1001f73ddb3c459c88073d9342b96ec3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:01.742574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e4cbeb08-964d-43c7-a732-85a16113cb17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1001f73ddb3c459c88073d9342b96ec3'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:01.754906 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4033/7993] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:24:01 2022] DELETE /identity/v3/users/1001f73ddb3c459c88073d9342b96ec3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:24:01.935516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:01.976429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46b88-e188-48b7-8875-246e4916593f service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.042954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.043647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.043921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.044185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] PATH_INFO: `/v3/projects/ca4a40ad2f664bd9bea4b640d6ab8fa1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.045330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4a40ad2f664bd9bea4b640d6ab8fa1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.047266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.094770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4a40ad2f664bd9bea4b640d6ab8fa1', 'request_id': 'req-f9705fe6-dd56-409c-86fb-626bc6cadd8a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:02.095916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:02.102113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4a40ad2f664bd9bea4b640d6ab8fa1', 'request_id': 'req-f9705fe6-dd56-409c-86fb-626bc6cadd8a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:02.109090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4a40ad2f664bd9bea4b640d6ab8fa1', 'request_id': 'req-f9705fe6-dd56-409c-86fb-626bc6cadd8a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:02.132927 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] The token cache is being invalidate because project ca4a40ad2f664bd9bea4b640d6ab8fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:24:02.133152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:02.133796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f9705fe6-dd56-409c-86fb-626bc6cadd8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:24:02.135307 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3961/7994] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:24:01 2022] DELETE /identity/v3/projects/ca4a40ad2f664bd9bea4b640d6ab8fa1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:24:02.140950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.141264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.141545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.189691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:02.190941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:02.250838 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4034/7995] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:02.259457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.259727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.260019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.281581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:02.282845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:02.291048 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3962/7996] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:02.295321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.305487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48b9ac-f494-4abb-bf1e-74b0e03c638a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.310417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.311081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.311356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.311623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.312260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:24:02.312557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.314180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.329734 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4035/7997] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:24:02.330981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.341364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f0122-07e6-4997-82cb-9084b4063d61 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.346197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.346876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.347133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.347418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.348045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.350316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.376861 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3963/7998] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:24:02 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:24:02.380996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.391807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c249-20ce-41c2-af1b-641e088deb86 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.583642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:02.595112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.595715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.595996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.596268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.596885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.598883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.621024 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4036/7999] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:24:02 2022] POST /identity/v3/users => generated 282 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:24:02.625252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.637034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb75781-14a4-4ab7-8c6f-811de7c99775 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.641709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.642356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.642624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.642898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.643795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.645402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.656398 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3964/8000] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:24:02.660512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.670705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07ec3-68f9-468d-a8d4-f06df2235af4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.675306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.675890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.676171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.676440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.677065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.678676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e7f44-6dcb-4efc-b804-5677f99567a2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.688936 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4037/8001] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:24:02.693001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.703518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b566-5271-43b2-a4e8-3b5754e57454 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.708081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.708720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.708999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.709273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] PATH_INFO: `/v3/projects/eb957f17053b49419fbbc09b8633628b/users/d7432bc67cac447391173eedccefbf9e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.723636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb957f17053b49419fbbc09b8633628b, user_id=d7432bc67cac447391173eedccefbf9e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.726289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.739238 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3965/8002] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:24:02 2022] PUT /identity/v3/projects/eb957f17053b49419fbbc09b8633628b/users/d7432bc67cac447391173eedccefbf9e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:24:02.744114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcff9-50dd-403f-b762-82208f57dc81 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.744481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcff9-50dd-403f-b762-82208f57dc81 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.744844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcff9-50dd-403f-b762-82208f57dc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.764936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3d5bcff9-50dd-403f-b762-82208f57dc81 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:02.766145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3d5bcff9-50dd-403f-b762-82208f57dc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:02.824453 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4038/8003] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:02.834676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:02.836770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:02.919707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3b036-527e-426e-890d-9aff1ea41862 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:02.923356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:02.923944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:02.924220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:02.924488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:02.934303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:02.935770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f4db7-f72c-41ea-ae63-ca3d36e000ab service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:02.945377 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3966/8004] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:24:07.204452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:07.204921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:07.205035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:07.237997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] MFA Rules not processed for user `d7432bc67cac447391173eedccefbf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:07.239328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:07.307634 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4039/8005] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:07.317921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:07.318476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:07.318476 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:07.340016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] MFA Rules not processed for user `d7432bc67cac447391173eedccefbf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:07.341253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:07.348201 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3967/8006] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:07.361266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:07.362662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:07.422567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:07.442661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9ccf-12bb-4160-b99c-b433c8eb8aa6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:07.448786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:07.449397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:07.449672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:07.449938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:07.458064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:07.459678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3866258-85dc-4fdc-9310-f5737a81eb57 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:07.469241 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4040/8007] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:24:07.721237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:07.722690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:07.737280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc94c-96e9-42a4-8ea4-8551ba3961e8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:07.740887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:07.741492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:07.741917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:07.742196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:07.743898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:07.760386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:07.763585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:07.775847 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3968/8008] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:24:17.465102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:17.769692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=3,keystone:SELECT=80,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:23.918282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4d490-4e8e-471f-8885-c7d757a624c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:23.918849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4d490-4e8e-471f-8885-c7d757a624c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:23.918963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4d490-4e8e-471f-8885-c7d757a624c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:23.940239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5af4d490-4e8e-471f-8885-c7d757a624c1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:23.941426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5af4d490-4e8e-471f-8885-c7d757a624c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:24.000221 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4041/8009] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:24.013180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:24.014609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:24.085574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa50f-287d-4fd9-81e5-c0f2611384c5 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:24.091217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:24.091817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:24.092098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:24.092371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:24.100433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:24.102025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:24.110120 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3969/8010] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:24:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:24:33.501796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ac5-67b2-425c-885c-488f0ce5bd30 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:33.502278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ac5-67b2-425c-885c-488f0ce5bd30 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:33.502403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ac5-67b2-425c-885c-488f0ce5bd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:33.521430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-26b01ac5-67b2-425c-885c-488f0ce5bd30 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:33.522656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-26b01ac5-67b2-425c-885c-488f0ce5bd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:33.529662 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4042/8011] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:33.541871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:33.556231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9c5d-4b74-454a-a3c8-bf414d568b5c service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:33.562242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:33.562859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:33.563134 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:33.563407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:33.571390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:33.572980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:33.581285 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3970/8012] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:24:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:24:33.995515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:33.995794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:33.996092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:34.016215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] MFA Rules not processed for user `d7432bc67cac447391173eedccefbf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:24:34.017426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:34.024632 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4043/8013] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:24:43.578136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=38 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:44.010753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=31 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:24:50.432983 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:24:50.445446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba631cf2-122b-4394-81ae-ee3f5f5a5517 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:24:50.449027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:24:50.450256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:24:50.450613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:24:50.450890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:24:50.452611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:24:50.468532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:24:50.469969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c26b3-3f23-48eb-b619-a9785c6884fa service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:24:50.483407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3971/8014] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:24:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:25:00.476024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:02.253565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:02.266426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5d193-fc0a-466b-a228-190a0844fa16 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:02.270377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:02.271009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:02.271301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:02.271558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:02.273335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:02.289783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:02.291274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:02.305063 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4044/8015] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:25:12.297663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:24.922394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12837aeb-e7be-469a-b9fe-9f36027ec78e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:24.922949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12837aeb-e7be-469a-b9fe-9f36027ec78e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:24.923063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12837aeb-e7be-469a-b9fe-9f36027ec78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:24.942411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12837aeb-e7be-469a-b9fe-9f36027ec78e None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:24.943644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12837aeb-e7be-469a-b9fe-9f36027ec78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:24.951779 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3972/8016] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:25:24.973703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:24.987844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fc12d-b50c-4320-b8dd-ab003017784b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:24.992912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:24.993532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:24.993807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:24.994074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:25.002857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:25.004422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:25.012454 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4045/8017] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:25:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:25:34.936893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:35.009047 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:35.186474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829ab6-483a-488b-be57-106acb35b370 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:35.186795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829ab6-483a-488b-be57-106acb35b370 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:35.187116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829ab6-483a-488b-be57-106acb35b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:35.207606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-39829ab6-483a-488b-be57-106acb35b370 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:35.208791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-39829ab6-483a-488b-be57-106acb35b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:35.216171 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3973/8018] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:25:35.228177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:35.241140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b608a13-01de-4b85-8cd1-08534f7b1681 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:35.246185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:35.247414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:35.247694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:35.247961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:35.255984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:35.257539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f02ead-3446-4fd1-a2b0-4c6e3c5fd7f0 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:35.266549 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4046/8019] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:25:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:25:35.700769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:35.700987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:35.701234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:35.721223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] MFA Rules not processed for user `d7432bc67cac447391173eedccefbf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:35.722408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:35.729292 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3974/8020] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:25:45.262450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:45.718657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:25:49.130154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa325a-ad96-4aca-9e37-f6beb83fb02f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:49.130706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa325a-ad96-4aca-9e37-f6beb83fb02f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:49.130811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa325a-ad96-4aca-9e37-f6beb83fb02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:49.151170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1bfa325a-ad96-4aca-9e37-f6beb83fb02f None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:49.152383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1bfa325a-ad96-4aca-9e37-f6beb83fb02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:49.159824 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4047/8021] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:25:49.172184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:49.187344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e09b2-4753-4b83-99eb-f6a409fbf922 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:49.192361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:49.192945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:49.193219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:49.193487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:49.201547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:49.204284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:49.212208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3975/8022] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:25:58.256528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289dea-1321-4393-b789-92fc12878bb3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:58.257075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289dea-1321-4393-b789-92fc12878bb3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:58.257075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289dea-1321-4393-b789-92fc12878bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:58.278958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d1289dea-1321-4393-b789-92fc12878bb3 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:58.281085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1289dea-1321-4393-b789-92fc12878bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:58.288253 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4048/8023] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:25:58.301527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:58.314807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb86b8-f814-411c-be54-a52f67ad2032 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:58.319879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:58.320473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:58.320754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:58.321028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:58.329849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:58.331568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:58.340607 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3976/8024] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:25:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 21:25:58.759195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:58.759474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:58.759744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:58.778701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] MFA Rules not processed for user `d7432bc67cac447391173eedccefbf9e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:25:58.780070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:58.786798 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4049/8025] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:25:58.911220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:25:58.925405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61129-9968-4e14-a37f-4d5ccc7e2d10 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:25:58.928905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:25:58.929493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:25:58.929767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:25:58.930045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:25:58.931752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:25:59.025857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:25:59.027383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:25:59.047822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3977/8026] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:25:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:00.006384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:00.022890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc8cb6-32c3-423e-a620-068b77ce8f34 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:00.027390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:00.028121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:00.028499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:00.028869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:00.031006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:00.063985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:00.065765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e5e48-b34d-4fe6-9c64-1bc5f8f23000 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:00.079937 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4050/8027] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:07.664344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:07.665868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:07.679979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308d8b2-09b5-49fd-a997-8f0f42aa88f0 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:07.684527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:07.685144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:07.685507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:07.685783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] PATH_INFO: `/v3/users/82c2160fcae947138ddd7b1b4df1e361` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:07.686930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c2160fcae947138ddd7b1b4df1e361)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:07.688841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:07.735924 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c2160fcae947138ddd7b1b4df1e361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:07.736815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c2160fcae947138ddd7b1b4df1e361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:07.744044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c2160fcae947138ddd7b1b4df1e361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:07.751452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c2160fcae947138ddd7b1b4df1e361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:07.752074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c2160fcae947138ddd7b1b4df1e361'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:07.769915 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3978/8028] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:07 2022] DELETE /identity/v3/users/82c2160fcae947138ddd7b1b4df1e361 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:07.774575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:07.775071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:07.775190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:07.794210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:07.795512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:07.856821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4051/8029] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:26:07.866588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:07.868018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:07.887311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:07.930336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f80ed-111d-42ab-83fa-3e564e50a449 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:07.934050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:07.934858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:07.935840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:07.936010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:07.945612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:07.947125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:07.957822 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3979/8030] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:08.210354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:08.211811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:08.225135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cab61-0fb5-46bd-8de2-02f78d7c3d97 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:08.229855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:08.230477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:08.230783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:08.231051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] PATH_INFO: `/v3/projects/a5c4c9ce4cb946a9a381271ab15aee09` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:08.233410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c4c9ce4cb946a9a381271ab15aee09)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:08.235475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:08.281999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c4c9ce4cb946a9a381271ab15aee09', 'request_id': 'req-9bf03459-e57f-4407-b499-1bb873db8010'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.287356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c4c9ce4cb946a9a381271ab15aee09', 'request_id': 'req-9bf03459-e57f-4407-b499-1bb873db8010'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.293384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c4c9ce4cb946a9a381271ab15aee09', 'request_id': 'req-9bf03459-e57f-4407-b499-1bb873db8010'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.306172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] The token cache is being invalidate because project a5c4c9ce4cb946a9a381271ab15aee09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:08.306454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.307116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.308568 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4052/8031] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:08 2022] DELETE /identity/v3/projects/a5c4c9ce4cb946a9a381271ab15aee09 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:08.313299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:08.324450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2890-4b64-476d-9820-3cc1d0c14beb service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:08.390909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:08.391881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:08.392235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:08.392584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] PATH_INFO: `/v3/users/3246fd815aa74d33b8eaeffcd719e2e1` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:08.407222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3246fd815aa74d33b8eaeffcd719e2e1)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:08.409339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:08.461406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3246fd815aa74d33b8eaeffcd719e2e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.462087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3246fd815aa74d33b8eaeffcd719e2e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.469150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3246fd815aa74d33b8eaeffcd719e2e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.476094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3246fd815aa74d33b8eaeffcd719e2e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.476758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3246fd815aa74d33b8eaeffcd719e2e1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.494520 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3980/8032] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:08 2022] DELETE /identity/v3/users/3246fd815aa74d33b8eaeffcd719e2e1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:08.675712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:08.677270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:08.692897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf03459-e57f-4407-b499-1bb873db8010 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:08.732988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:08.753133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:08.753759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:08.754041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:08.754312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] PATH_INFO: `/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:08.765049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f4287f4d0b4f2281d6c40edcba11fd)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:08.767253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:08.813449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f4287f4d0b4f2281d6c40edcba11fd', 'request_id': 'req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.818782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f4287f4d0b4f2281d6c40edcba11fd', 'request_id': 'req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.825707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f4287f4d0b4f2281d6c40edcba11fd', 'request_id': 'req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.840265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] The token cache is being invalidate because project a4f4287f4d0b4f2281d6c40edcba11fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:08.840556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.841213 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:08.843670 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4053/8033] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:08 2022] DELETE /identity/v3/projects/a4f4287f4d0b4f2281d6c40edcba11fd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:26:18.491027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=77,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:18.836327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:28.860383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:28.862012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:28.877463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418e74-e566-40f0-9a5e-c84f436ae632 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:28.942352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:28.943013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:28.943310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:28.943581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] PATH_INFO: `/v3/users/d7432bc67cac447391173eedccefbf9e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:28.944694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7432bc67cac447391173eedccefbf9e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:28.946638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:28.995413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7432bc67cac447391173eedccefbf9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:28.996052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7432bc67cac447391173eedccefbf9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.004654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7432bc67cac447391173eedccefbf9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.011827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7432bc67cac447391173eedccefbf9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.012319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-10d26ffb-885f-46d1-98c2-268ceffc76f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7432bc67cac447391173eedccefbf9e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.030916 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3981/8034] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:28 2022] DELETE /identity/v3/users/d7432bc67cac447391173eedccefbf9e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:29.210237 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.211707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:29.225271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c2eb-25db-41e0-92f1-34ae5e5abf3b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.284196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.284832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.285108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.285380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] PATH_INFO: `/v3/projects/eb957f17053b49419fbbc09b8633628b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.286478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb957f17053b49419fbbc09b8633628b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.288405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.333315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb957f17053b49419fbbc09b8633628b', 'request_id': 'req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.338737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb957f17053b49419fbbc09b8633628b', 'request_id': 'req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.345276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb957f17053b49419fbbc09b8633628b', 'request_id': 'req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.357242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] The token cache is being invalidate because project eb957f17053b49419fbbc09b8633628b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:29.357574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.359458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0bd655f8-0030-4aeb-ba7a-dd740c2e1b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:29.361075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4054/8035] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:29 2022] DELETE /identity/v3/projects/eb957f17053b49419fbbc09b8633628b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:29.366053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.366339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.366650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.405902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:29.407053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:29.466093 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3982/8036] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:29.475310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.475597 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.475875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.496176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:29.497380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:29.505395 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4055/8037] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:29.509564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.515608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:29.520271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2437ecd-7da8-4106-9ce0-ab03be43d895 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.525989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.526670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.527128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.527405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.528057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:26:29.528362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.530554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.542022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3983/8038] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:26:29.546792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.555888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ed187-1116-4711-9a7e-8b067cbedd1d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.560480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.561154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.561448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.561696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.562320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.564241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.601573 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4056/8039] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:29 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:29.605642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.615658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf5c9b-cc55-4fca-b825-ac7cf5cf2e14 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.675291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.675821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.676097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.676372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.677001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.678953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.698799 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3984/8040] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:29 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:29.703634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.713759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917196ec-d89a-4954-bdce-a20a5655d840 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.718919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.719503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.719776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.720040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.720663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.722206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.731809 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4057/8041] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:29.736012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.746043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a7cf-f12a-400d-9242-4345000d003d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.755259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.755864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.756141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.756440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.757073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.758982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e27e0-bc5e-462d-9594-3d6047847103 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.770653 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3985/8042] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:29.774801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.788672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f0bac-f8d4-482a-b7e5-f1166c897724 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:29.795896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:29.796536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.796812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.797081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] PATH_INFO: `/v3/projects/4a738f6c7d7943ed89991f262ebf8faf/users/b9541e94f4a142a6bf275d6d4b5c34b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.811565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a738f6c7d7943ed89991f262ebf8faf, user_id=b9541e94f4a142a6bf275d6d4b5c34b6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:29.814196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:29.827007 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4058/8043] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:29 2022] PUT /identity/v3/projects/4a738f6c7d7943ed89991f262ebf8faf/users/b9541e94f4a142a6bf275d6d4b5c34b6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:29.837041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:29.837349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:29.837624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:29.865702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] MFA Rules not processed for user `b9541e94f4a142a6bf275d6d4b5c34b6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:29.866939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:29.931757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3986/8044] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:29.943976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:29.945706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:29.966862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:30.024965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13353b-cd6f-4d06-b20c-a9e4a3f78262 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:30.030653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:30.031245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.031520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.031786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.039463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:30.041067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f95b0-6eaf-4ee2-9c3d-8e3d1d5ab037 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:30.049806 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4059/8045] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:30.237136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:30.298677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22a750-7938-4627-91da-624d0db4c924 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:30.303462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:30.304097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.304377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.304739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] PATH_INFO: `/v3/users/b9541e94f4a142a6bf275d6d4b5c34b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.305879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9541e94f4a142a6bf275d6d4b5c34b6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:30.307829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:30.309391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:30.364047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9541e94f4a142a6bf275d6d4b5c34b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.364829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9541e94f4a142a6bf275d6d4b5c34b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.374563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9541e94f4a142a6bf275d6d4b5c34b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.382411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9541e94f4a142a6bf275d6d4b5c34b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.383148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9541e94f4a142a6bf275d6d4b5c34b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.403440 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3987/8046] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:30 2022] DELETE /identity/v3/users/b9541e94f4a142a6bf275d6d4b5c34b6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:30.408060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.408547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.408662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.428298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:30.429499 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:30.489862 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4060/8047] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:30.500211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:30.501672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:30.560777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e121e-95b4-47be-ad69-cc0978302430 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:30.565293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:30.565885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.566190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.566460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.575469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:30.576972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcab747-fb92-49d1-adcd-a7e819db85b0 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:30.586379 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3988/8048] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:26:30.846681 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:30.848115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:30.860865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf38581-79aa-40b3-bdc7-f5d18f3e2c22 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:30.865591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:30.866224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.866539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.866808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] PATH_INFO: `/v3/projects/4a738f6c7d7943ed89991f262ebf8faf` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.867941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a738f6c7d7943ed89991f262ebf8faf)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:30.869846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:30.917769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a738f6c7d7943ed89991f262ebf8faf', 'request_id': 'req-aa018fc8-6390-4414-9ebd-fe1c121f349e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.923835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a738f6c7d7943ed89991f262ebf8faf', 'request_id': 'req-aa018fc8-6390-4414-9ebd-fe1c121f349e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.929883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a738f6c7d7943ed89991f262ebf8faf', 'request_id': 'req-aa018fc8-6390-4414-9ebd-fe1c121f349e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.942424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] The token cache is being invalidate because project 4a738f6c7d7943ed89991f262ebf8faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:30.942746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.943385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-aa018fc8-6390-4414-9ebd-fe1c121f349e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:30.944851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4061/8049] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:30 2022] DELETE /identity/v3/projects/4a738f6c7d7943ed89991f262ebf8faf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:30.949859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:30.950360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:30.950466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:30.970799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:30.971930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:31.031092 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3989/8050] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:31.038620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.038909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.039186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.059244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:31.060447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:31.068685 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4062/8051] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:31.072815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.083348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b5e2-3398-4d7e-9c49-e61fc6dafc4c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.087955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.088546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.088829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.089105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.089748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:26:31.090052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.091694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.103613 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3990/8052] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:26:31.107798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.117119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb105b-0866-4fda-9c50-59cb281ee363 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.121695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.122278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.122579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.122851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.123479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.125389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.146360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:31.163339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4063/8053] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:31 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:31.164587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.174963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff8f2-8853-4d73-ac3b-da90d447b33e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.179622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:31.231644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.232160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.232432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.232806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.233415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.235349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.255641 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3991/8054] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:31 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:31.259707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.270689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a804-e1a7-4076-b630-3a643f35ea71 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.275232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.275813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.276089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.276353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.276978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.278580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.289488 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4064/8055] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:31.293214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.302995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cbd2f-ebae-44a7-92e2-02f241af8ed6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.307576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.308157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.308448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.308697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.309312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.311198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6e850-fb5f-4415-9cff-6db732f00227 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.322061 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3992/8056] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:31.326185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.336710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214b8e1-0e81-460f-8757-21b5e80a7c93 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.341503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.342822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.343091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.343358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] PATH_INFO: `/v3/projects/9124ddf2f38849c2a674272858e07dac/users/99477c81a4d14dd685eef916ae5af9c7/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.357971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9124ddf2f38849c2a674272858e07dac, user_id=99477c81a4d14dd685eef916ae5af9c7, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.360630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.372961 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4065/8057] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:31 2022] PUT /identity/v3/projects/9124ddf2f38849c2a674272858e07dac/users/99477c81a4d14dd685eef916ae5af9c7/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:26:31.383937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.384239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.384490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.414234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] MFA Rules not processed for user `99477c81a4d14dd685eef916ae5af9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:31.415469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:31.479393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3993/8058] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:31.490722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.492341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:31.568838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090be2-c813-412b-897d-2dd19058def6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.575270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.575875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.576155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.577273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.585202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.586805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc37a3-0504-47ec-9474-47149c111d53 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.595764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4066/8059] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:31.779177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:31.840609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce412952-0c86-4419-8cae-2d420341be58 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:31.845215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:31.845829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.846104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.846373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] PATH_INFO: `/v3/users/99477c81a4d14dd685eef916ae5af9c7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.847498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=99477c81a4d14dd685eef916ae5af9c7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:31.849407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:31.870184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:31.895930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99477c81a4d14dd685eef916ae5af9c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:31.896576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99477c81a4d14dd685eef916ae5af9c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:31.904806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99477c81a4d14dd685eef916ae5af9c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:31.911936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99477c81a4d14dd685eef916ae5af9c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:31.912581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99477c81a4d14dd685eef916ae5af9c7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:31.930643 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3994/8060] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:31 2022] DELETE /identity/v3/users/99477c81a4d14dd685eef916ae5af9c7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:31.935158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:31.935478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:31.936331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:31.955899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:31.957150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:32.016127 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4067/8061] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:32.025724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.027185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:32.086405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb7330-d131-43ee-b13e-ee4990acaaaa admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.090132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.090757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.091031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.091295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.100380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.101801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73a4e1-25a0-464d-95b1-0cf03d06c925 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.111982 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3995/8062] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:32.389706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.391774 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:32.406658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05948c-a307-4c34-b61f-979c1d077ac9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.411408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.412040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.412318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.412605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] PATH_INFO: `/v3/projects/9124ddf2f38849c2a674272858e07dac` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.413718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9124ddf2f38849c2a674272858e07dac)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.415650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.464842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:32.469751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9124ddf2f38849c2a674272858e07dac', 'request_id': 'req-dbecbf54-0329-4f49-a721-964939a0db97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:32.475893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9124ddf2f38849c2a674272858e07dac', 'request_id': 'req-dbecbf54-0329-4f49-a721-964939a0db97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:32.482062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9124ddf2f38849c2a674272858e07dac', 'request_id': 'req-dbecbf54-0329-4f49-a721-964939a0db97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:32.494381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] The token cache is being invalidate because project 9124ddf2f38849c2a674272858e07dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:32.495011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:32.495292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-dbecbf54-0329-4f49-a721-964939a0db97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:32.496767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4068/8063] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:32 2022] DELETE /identity/v3/projects/9124ddf2f38849c2a674272858e07dac => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:32.501762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.502045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.502320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.521539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:32.522692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:32.580397 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3996/8064] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:32.588155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.588440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.588715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.609074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:32.610271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:32.618116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4069/8065] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:26:32.622230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.632084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f229-34cc-4540-addd-f0f6262b2a35 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.636820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.637404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.637681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.637950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.638630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:26:32.639473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.641329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.653035 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3997/8066] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:26:32.657977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.667516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d6f80-2be9-4254-a082-812b2b71fd20 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.672440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.673038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.673316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.673585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.674205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.676135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.703179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4070/8067] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:32 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:32.707352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.718095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ca6eb-5ae9-4c70-8540-22e6238a1286 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.742513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:32.776249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.776762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.777044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.777320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.777956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.780601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.800274 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3998/8068] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:32 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:32.804338 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.815001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cc703-69b7-440b-9ecd-3364f8ff3a25 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.819647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.820268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.820660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.820937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.821566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.823351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.833870 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4071/8069] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:32.837634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.848423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f58caa-e0ca-4aaf-9a11-c0d05c7721b4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.853156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.853747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.854029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.854305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.854974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.856567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c5f7-53e1-4511-90f4-f3ab68175d55 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.867289 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 3999/8070] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:32.871339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:32.881496 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35908c-ecd7-47fa-b7ad-85dea4065396 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:32.886112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:32.886794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.887073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.887366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] PATH_INFO: `/v3/projects/2ec88f37776f490794836a3796812fb6/users/7a34302354c7485da79fac333dbe361e/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.902373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec88f37776f490794836a3796812fb6, user_id=7a34302354c7485da79fac333dbe361e, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:32.905701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:32.918218 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4072/8071] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:32 2022] PUT /identity/v3/projects/2ec88f37776f490794836a3796812fb6/users/7a34302354c7485da79fac333dbe361e/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:32.923600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e494-8fb1-4a43-a6c8-47db9dae0182 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:32.923891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e494-8fb1-4a43-a6c8-47db9dae0182 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:32.924169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e494-8fb1-4a43-a6c8-47db9dae0182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:32.943377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-c7a8e494-8fb1-4a43-a6c8-47db9dae0182 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:32.944589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c7a8e494-8fb1-4a43-a6c8-47db9dae0182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:33.003064 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4000/8072] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:33.012602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:33.014980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:33.073758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a4a8a-c981-4dc3-b15d-02b48ce30587 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:33.078307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:33.078977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:33.079260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:33.079533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:33.088748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:33.090176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:33.099594 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4073/8073] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:37.528197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:37.528759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:37.528759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:37.558080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] MFA Rules not processed for user `7a34302354c7485da79fac333dbe361e`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:37.559331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:37.635445 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4001/8074] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:37.650284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:37.652106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:37.736648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111ea89-9874-477a-89ee-47da7e9bf087 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:37.742008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:37.742635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:37.742910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:37.743175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:37.749750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:37.752099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:37.753966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:37.765075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4074/8075] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:37 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:43.540639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:43.552421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9331db6-2ec1-42ac-85fe-880d71db0416 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:43.557189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:43.557977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:43.558269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:43.558572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] PATH_INFO: `/v3/users/7a34302354c7485da79fac333dbe361e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:43.559745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a34302354c7485da79fac333dbe361e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:43.562766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:43.605610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:43.624030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a34302354c7485da79fac333dbe361e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.624741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a34302354c7485da79fac333dbe361e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.636459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a34302354c7485da79fac333dbe361e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.646698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a34302354c7485da79fac333dbe361e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.647521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb6ac181-3c32-4581-bafe-78bb6f92b370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a34302354c7485da79fac333dbe361e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.668244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4002/8076] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:43 2022] DELETE /identity/v3/users/7a34302354c7485da79fac333dbe361e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:43.859822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:43.873557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10dd05-380a-4618-837b-f385f21abed7 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:43.933879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:43.934520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:43.934811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:43.935855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] PATH_INFO: `/v3/projects/2ec88f37776f490794836a3796812fb6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:43.937184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec88f37776f490794836a3796812fb6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:43.939112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:43.985333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec88f37776f490794836a3796812fb6', 'request_id': 'req-72182a98-fb59-41c7-a88e-d173e372ccd2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.990878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec88f37776f490794836a3796812fb6', 'request_id': 'req-72182a98-fb59-41c7-a88e-d173e372ccd2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:43.997089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec88f37776f490794836a3796812fb6', 'request_id': 'req-72182a98-fb59-41c7-a88e-d173e372ccd2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.010394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] The token cache is being invalidate because project 2ec88f37776f490794836a3796812fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:44.010748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.011375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-72182a98-fb59-41c7-a88e-d173e372ccd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.012892 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4075/8077] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:43 2022] DELETE /identity/v3/projects/2ec88f37776f490794836a3796812fb6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:44.018223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.018520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.018793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.038293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:44.039447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:44.098887 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4003/8078] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:26:44.106709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.107889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.108254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.129548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:44.130793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:44.138799 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4076/8079] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:44.143270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.152943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99014-7a46-4ce0-9e4b-c36bed38d46c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.157826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.158413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.158727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.159006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.159648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:26:44.159948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.161610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.173829 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4004/8080] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:26:44.178165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.188704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa3e8-75ba-40ac-8040-82c0e5efa6c1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.193411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.193997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.194275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.194583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.195216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.197094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.223784 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4077/8081] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:26:44.228044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.239162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155becb-2862-44a6-abc2-4c37001adb1f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.295190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.295699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.295980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.296256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.296902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.298863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.319645 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4005/8082] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/users => generated 280 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:44.323686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.334199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc29bb7-6e03-40b7-9a30-a172628a9f72 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.338846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.339439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.339721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.339975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.340605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.342949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.353023 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4078/8083] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:44.357549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.370641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49eabd-d780-41ca-91e1-a2c4d6636531 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.376025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.376683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.378215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.378556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.379253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.380886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b90f4-1eb3-4085-bd33-a9f3a8526590 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.392426 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4006/8084] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:44.396760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.408650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d459a-d082-4b7a-80a3-2075207944a5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.413495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.414222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.414614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.414990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] PATH_INFO: `/v3/projects/4a5887ad433b4f8c8627a41383e5e15b/users/839b754f9b3147daa386158b9fe2b42d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.433052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5887ad433b4f8c8627a41383e5e15b, user_id=839b754f9b3147daa386158b9fe2b42d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.436348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.450310 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4079/8085] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:44 2022] PUT /identity/v3/projects/4a5887ad433b4f8c8627a41383e5e15b/users/839b754f9b3147daa386158b9fe2b42d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:26:44.460744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.461233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.461580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.489393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:44.492861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] MFA Rules not processed for user `839b754f9b3147daa386158b9fe2b42d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:44.494715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:44.559701 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4007/8086] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:44.571054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.572755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:44.599812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:44.651172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1fdd0-30c9-4af6-9493-58cfaf8c0711 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.658155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.658809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.659197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.659474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.667511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.669107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac4d81-e66a-4358-bb2d-2e63ddcc836f service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.678255 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4080/8087] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:44.722833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.785296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308ac48-9a52-41f9-8f6d-61583f09a48c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:44.789906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:44.790559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.790845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.791120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] PATH_INFO: `/v3/users/839b754f9b3147daa386158b9fe2b42d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.792252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=839b754f9b3147daa386158b9fe2b42d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:44.794175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:44.840208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839b754f9b3147daa386158b9fe2b42d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.840865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839b754f9b3147daa386158b9fe2b42d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.849433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839b754f9b3147daa386158b9fe2b42d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.856829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839b754f9b3147daa386158b9fe2b42d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.857530 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839b754f9b3147daa386158b9fe2b42d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:44.878857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4008/8088] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:26:44 2022] DELETE /identity/v3/users/839b754f9b3147daa386158b9fe2b42d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:44.883382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:44.883896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:44.884013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:44.905409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:44.906674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:44.964036 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4081/8089] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:44.973909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:44.975348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:45.001714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:45.039512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc737a-3f3e-46e4-92d6-89d2bb97db98 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.043070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.043666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.043942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.044233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.053153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.054619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960d462-58b6-4998-bd31-8ab2869c35a1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.064796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4009/8090] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:45.331802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.333406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:45.348007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b895f72-7daa-41cf-9845-e0e2475ba62f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.352696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.353323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.353600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.353868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] PATH_INFO: `/v3/projects/4a5887ad433b4f8c8627a41383e5e15b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.355019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5887ad433b4f8c8627a41383e5e15b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.357513 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.407333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5887ad433b4f8c8627a41383e5e15b', 'request_id': 'req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:45.412657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5887ad433b4f8c8627a41383e5e15b', 'request_id': 'req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:45.418901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5887ad433b4f8c8627a41383e5e15b', 'request_id': 'req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:45.432228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] The token cache is being invalidate because project 4a5887ad433b4f8c8627a41383e5e15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:26:45.432522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:45.433246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5bab1ad3-1ed2-4f9d-8efd-fe05c82cd07e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:26:45.434838 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4082/8091] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:26:45 2022] DELETE /identity/v3/projects/4a5887ad433b4f8c8627a41383e5e15b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:45.440680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.440963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.441237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.460814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:45.461999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:45.522912 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4010/8092] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:45.531957 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.532243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.532509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.552905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:45.554112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:45.562884 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4083/8093] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:45.567133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.577438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636be11c-3b4d-4261-b8dc-09d11a233462 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.582414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.583039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.583318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.583615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.584284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:26:45.584588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.586213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.597723 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4011/8094] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:26:45.602164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.612243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b54c74-f378-44b8-a1c1-4ff674c25ee7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.616882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.617469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.617746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.618006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.618679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.620563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.649860 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4084/8095] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:45 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:26:45.654178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.664961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac2372-ed69-41cc-8b45-db311bda5cbb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.721579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.722175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.722453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.722831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.723462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.725347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.747017 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4012/8096] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:45 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:45.751236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.756728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:45.762753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b12a3-2f1d-49b4-8faa-d0400db85b0d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.768901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.769492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.769802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.770069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.770734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.772371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.783244 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4085/8097] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:45.787094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.797664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eabc0b-829f-4288-97eb-f7d40fc32379 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.802651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.803270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.803575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.803850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.804477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.806068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2b0de-3456-49c6-b2d1-362f24360d19 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.816666 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4013/8098] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:45.820858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.830832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551308c-e056-42bf-b768-b51c5c51a2e9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:45.835749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:45.836398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.836677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.836977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] PATH_INFO: `/v3/projects/194a1bb2cb3a4c48865a80e37c18403a/users/327909bb355a4e38829344cff7c78c97/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.852080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=194a1bb2cb3a4c48865a80e37c18403a, user_id=327909bb355a4e38829344cff7c78c97, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:45.854775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:45.867876 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4086/8099] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:45 2022] PUT /identity/v3/projects/194a1bb2cb3a4c48865a80e37c18403a/users/327909bb355a4e38829344cff7c78c97/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:45.872959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76658e-b96b-4602-9130-6191170c3d6e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:45.873246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76658e-b96b-4602-9130-6191170c3d6e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:45.873523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76658e-b96b-4602-9130-6191170c3d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:45.892716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5f76658e-b96b-4602-9130-6191170c3d6e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:45.893915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f76658e-b96b-4602-9130-6191170c3d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:45.920542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:45.953768 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4014/8100] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:45.963564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:45.965242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:46.024756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5ab21-8997-4583-b25d-7ccaa8448f5a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:46.028489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:46.029594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:46.029876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:46.030151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:46.039128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:46.040553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:46.050558 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4087/8101] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:26:50.090301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.090844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.091003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.129181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] MFA Rules not processed for user `327909bb355a4e38829344cff7c78c97`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:50.130423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:50.195616 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4015/8102] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:50.200340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.211158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda30dd5-759c-43e8-90b7-eb2d012dfe92 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.216646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.217524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.217798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.218062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.218932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.220867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.251807 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4088/8103] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:26:50 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:50.255659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.267520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a0306-a853-4354-a8c4-73ef800ca125 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.336599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.337293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.337579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.337850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.338553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.340654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.364979 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4016/8104] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:26:50 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:26:50.369296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.384475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc99816-b7a1-44c4-948d-9c093bd7e938 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.389951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.390892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.391172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.391439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.392177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.393804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.406787 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4089/8105] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:50.411301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.430273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882b5a2-7fb1-41f3-9f48-1fef9d41667a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.439344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.440111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.440478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.440839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] PATH_INFO: `/v3/projects/899072015b37456494653c118a418a4d/users/ce624396e2e648399f5b52b579905845/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.457577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=899072015b37456494653c118a418a4d, user_id=ce624396e2e648399f5b52b579905845, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.460298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.480251 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4017/8106] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:50 2022] PUT /identity/v3/projects/899072015b37456494653c118a418a4d/users/ce624396e2e648399f5b52b579905845/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:26:50.484649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.495311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf8b9-55cf-470a-bb4f-941cc6e69ba7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.571586 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.572521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.572801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.573067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.573709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.576023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e11da-e637-48dd-a78d-8b8fcdbf3fe6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.594764 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4090/8107] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:26:50.596008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:50.608483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46fc38-6a5d-4e51-8a6c-6327493770cc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:50.613497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:50.614173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:50.614454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:50.614761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] PATH_INFO: `/v3/projects/899072015b37456494653c118a418a4d/users/ce624396e2e648399f5b52b579905845/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:50.616703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=899072015b37456494653c118a418a4d, user_id=ce624396e2e648399f5b52b579905845, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:50.619558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:50.641442 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4018/8108] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:26:50 2022] PUT /identity/v3/projects/899072015b37456494653c118a418a4d/users/ce624396e2e648399f5b52b579905845/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:26:55.195567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:55.196064 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:55.196247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:55.225164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] MFA Rules not processed for user `ce624396e2e648399f5b52b579905845`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:26:55.226421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:55.248563 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:55.299031 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4091/8109] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:26:55.311237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:26:55.312994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:26:55.328497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:26:55.393245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf40d6b-c5e4-4a1a-a0d9-7dcfe5e485ee admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:26:55.396957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:26:55.397593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:26:55.397861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:26:55.398137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:26:55.463942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:26:55.465397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:26:55.475455 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4019/8110] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:03.757317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:03.821668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f002c8-3d7a-417d-9cb6-7d25346e3372 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:03.826920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:03.827543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:03.827824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:03.828094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] PATH_INFO: `/v3/users/327909bb355a4e38829344cff7c78c97` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:03.829214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327909bb355a4e38829344cff7c78c97)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:03.831169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:03.878450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327909bb355a4e38829344cff7c78c97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:03.879065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327909bb355a4e38829344cff7c78c97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:03.886439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327909bb355a4e38829344cff7c78c97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:03.899986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327909bb355a4e38829344cff7c78c97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:03.900649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327909bb355a4e38829344cff7c78c97'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:03.913108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4092/8111] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:03 2022] DELETE /identity/v3/users/327909bb355a4e38829344cff7c78c97 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:04.083964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.097026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e57ff-7547-4508-b597-b281ce1af532 service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:04.155852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:04.156477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.156755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.157027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] PATH_INFO: `/v3/projects/194a1bb2cb3a4c48865a80e37c18403a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.158160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=194a1bb2cb3a4c48865a80e37c18403a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:04.160108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:04.208007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194a1bb2cb3a4c48865a80e37c18403a', 'request_id': 'req-229b7202-6443-48b8-b998-f8e73caf21bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.213338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194a1bb2cb3a4c48865a80e37c18403a', 'request_id': 'req-229b7202-6443-48b8-b998-f8e73caf21bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.220552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194a1bb2cb3a4c48865a80e37c18403a', 'request_id': 'req-229b7202-6443-48b8-b998-f8e73caf21bb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.227554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:04.233334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] The token cache is being invalidate because project 194a1bb2cb3a4c48865a80e37c18403a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:04.233893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.235004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.236868 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4020/8112] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:27:04 2022] DELETE /identity/v3/projects/194a1bb2cb3a4c48865a80e37c18403a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:04.240890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.242280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:04.256013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8c92-d8ea-40ab-b0a9-cd1fa2b30611 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:04.288548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:04.314534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:04.315170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.315453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.315724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] PATH_INFO: `/v3/users/ce624396e2e648399f5b52b579905845` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.316936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce624396e2e648399f5b52b579905845)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:04.320021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:04.367370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce624396e2e648399f5b52b579905845'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.368039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce624396e2e648399f5b52b579905845'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.376361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce624396e2e648399f5b52b579905845'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.389174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce624396e2e648399f5b52b579905845'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.389836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-962b0d94-134c-4503-9b3c-9e2b1c7a54d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce624396e2e648399f5b52b579905845'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.402846 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4093/8113] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:04 2022] DELETE /identity/v3/users/ce624396e2e648399f5b52b579905845 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:04.584254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.585722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:04.600641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7202-6443-48b8-b998-f8e73caf21bb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:04.664556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:04.665191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.665471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.665746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] PATH_INFO: `/v3/projects/899072015b37456494653c118a418a4d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.666906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=899072015b37456494653c118a418a4d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:04.669521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:04.713752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899072015b37456494653c118a418a4d', 'request_id': 'req-4242a118-7218-4319-8db8-65cff108b20b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.719783 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899072015b37456494653c118a418a4d', 'request_id': 'req-4242a118-7218-4319-8db8-65cff108b20b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.726940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899072015b37456494653c118a418a4d', 'request_id': 'req-4242a118-7218-4319-8db8-65cff108b20b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.738465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] The token cache is being invalidate because project 899072015b37456494653c118a418a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:04.738786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.739426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4242a118-7218-4319-8db8-65cff108b20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:04.740891 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4021/8114] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:27:04 2022] DELETE /identity/v3/projects/899072015b37456494653c118a418a4d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:04.745836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.746114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.746426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.766235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:04.767478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:04.827337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4094/8115] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:04.836248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.836534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.836811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.856999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:04.858197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:04.866465 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4022/8116] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:04.870779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.880698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de94a7-445a-469a-84d9-d8b22d40f3d2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:04.885322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:04.885914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.886193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.886462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.887646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:27:04.887943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:04.889547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:04.901182 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4095/8117] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:27:04.906019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.915413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa152da-c198-45d0-9e5f-62e263d2c889 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:04.920205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:04.920793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:04.921072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:04.921342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:04.921955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:04.923926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:04.952633 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4023/8118] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:27:04 2022] POST /identity/v3/projects => generated 395 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:04.956877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:04.967901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245a90-c11a-4d9c-84cc-4246c5686ea8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.009165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:05.025996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.026647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.026933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.027206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.028497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.030416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.050399 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4096/8119] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:04 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:05.054734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.065162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb943b-0dd2-4fd8-956c-af1e0966560f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.069910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.070524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.070815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.071087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.071718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.073457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.083878 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4024/8120] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:05.087722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.098270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4769d8a-5eca-4bdc-ac72-2609570ccfae admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.102925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.103516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.103797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.104075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.104707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.106310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d390e-9fbb-4027-ae9e-ae2459893883 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.116759 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4097/8121] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:05.120702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.130962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c26247-05aa-4c30-8fa9-280cbdaad68d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.135661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.136301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.136586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.137407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] PATH_INFO: `/v3/projects/d6b8f825fc1a4dfdb47f5b07c729153b/users/4fbe0c5b4e694811b63f0fcad2099add/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.152197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b8f825fc1a4dfdb47f5b07c729153b, user_id=4fbe0c5b4e694811b63f0fcad2099add, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.155524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4e041-394a-4eb5-9a2d-bb8a9f3fcbf8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.169407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4025/8122] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:05 2022] PUT /identity/v3/projects/d6b8f825fc1a4dfdb47f5b07c729153b/users/4fbe0c5b4e694811b63f0fcad2099add/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:05.179536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.179823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.180094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.208334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.209568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.273057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4098/8123] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.283964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718893-6ed0-4c22-b49d-7f3ae6889b34 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.284246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718893-6ed0-4c22-b49d-7f3ae6889b34 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.284519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718893-6ed0-4c22-b49d-7f3ae6889b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.303586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-26718893-6ed0-4c22-b49d-7f3ae6889b34 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.304784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26718893-6ed0-4c22-b49d-7f3ae6889b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.312332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4026/8124] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.325249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.326738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.409285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735b6f2-c9e6-44dd-bc37-020a829c0335 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.415033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.415926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.416202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.416472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.424598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.426160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f40156-b836-4338-ab12-da0e277e6e8b service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.435689 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4099/8125] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:05 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:05.460500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0568b6-5d63-4157-99f9-5f4906c98049 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.460785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0568b6-5d63-4157-99f9-5f4906c98049 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.461062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0568b6-5d63-4157-99f9-5f4906c98049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.480193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ba0568b6-5d63-4157-99f9-5f4906c98049 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.481471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba0568b6-5d63-4157-99f9-5f4906c98049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.488421 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4027/8126] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.514029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be56672-c972-4da6-ac59-d2ef1a10ce58 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.514315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be56672-c972-4da6-ac59-d2ef1a10ce58 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.514639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be56672-c972-4da6-ac59-d2ef1a10ce58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.533780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8be56672-c972-4da6-ac59-d2ef1a10ce58 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.535025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8be56672-c972-4da6-ac59-d2ef1a10ce58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.542588 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4100/8127] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.567845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.568133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.568414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.588517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.589750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.596743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4028/8128] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.630356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a2c7-8898-480e-bbe2-ad06933d50d0 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.630676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a2c7-8898-480e-bbe2-ad06933d50d0 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.630951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a2c7-8898-480e-bbe2-ad06933d50d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.651051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0b75a2c7-8898-480e-bbe2-ad06933d50d0 None None] MFA Rules not processed for user `4fbe0c5b4e694811b63f0fcad2099add`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.652334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b75a2c7-8898-480e-bbe2-ad06933d50d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.659909 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4101/8129] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:05.687249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.693070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:05.749722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56f6f8-a4b8-4d64-b177-2da3dd116a23 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:05.754313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:05.754958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.755237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.755506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] PATH_INFO: `/v3/users/4fbe0c5b4e694811b63f0fcad2099add` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.756613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbe0c5b4e694811b63f0fcad2099add)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:05.758557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:05.804766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbe0c5b4e694811b63f0fcad2099add'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:05.805433 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbe0c5b4e694811b63f0fcad2099add'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:05.813362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbe0c5b4e694811b63f0fcad2099add'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:05.820376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbe0c5b4e694811b63f0fcad2099add'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:05.821089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbe0c5b4e694811b63f0fcad2099add'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:05.838258 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4029/8130] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:05 2022] DELETE /identity/v3/users/4fbe0c5b4e694811b63f0fcad2099add => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:05.842981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:05.843457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:05.843562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:05.864276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:05.865489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.883903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:05.925515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4102/8131] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:27:05.935853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:05.937299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:05.997775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8aaea-03af-4415-bfac-711ae1fe825c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.001460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.002048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.002321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.002617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.011418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.013140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108183b4-0ac6-482d-8d89-42080b7ee85c service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.022695 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4030/8132] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:06.288600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.290129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:06.303988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e5554-f30f-4c17-8764-eb028bdb1c93 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.309879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.310684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.310965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.311240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] PATH_INFO: `/v3/projects/d6b8f825fc1a4dfdb47f5b07c729153b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.312434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b8f825fc1a4dfdb47f5b07c729153b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.314330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.362979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b8f825fc1a4dfdb47f5b07c729153b', 'request_id': 'req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:06.368564 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b8f825fc1a4dfdb47f5b07c729153b', 'request_id': 'req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:06.374710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b8f825fc1a4dfdb47f5b07c729153b', 'request_id': 'req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:06.387470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] The token cache is being invalidate because project d6b8f825fc1a4dfdb47f5b07c729153b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:06.387753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:06.388373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7dfe26d5-622c-4fc8-b88b-3f2f21c1795e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:06.389842 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4103/8133] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:27:06 2022] DELETE /identity/v3/projects/d6b8f825fc1a4dfdb47f5b07c729153b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:06.396018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.396521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.396625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.417124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:06.418428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:06.438849 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:06.479848 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4031/8134] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:06.487350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.487631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.487902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.507856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:06.509063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:06.518311 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4104/8135] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:06.522663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.533275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d5fd7-b699-4476-978c-ea38c75b1614 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.538055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.538691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.538971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.539243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.539884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:27:06.540184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.541779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.553396 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4032/8136] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:27:06.557739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.567795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba2fa3-0575-4ea2-883d-2bc0c50af1e2 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.572548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.573147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.573430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.573700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.574860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.576760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.604066 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4105/8137] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:27:06 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:06.608261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.618524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57e4e3-7bff-4c06-89d0-b6738419bd93 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.681897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.682520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.682806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.683080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.684222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.686138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.707248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4033/8138] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:06 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:06.711402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.721792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad543580-b9ac-4aee-ba6e-08878e5f65cc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.726460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.727089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.727367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.727644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.728277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.729907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.739702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4106/8139] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:06.743555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.754126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb02f-fef0-4082-b1b0-e1cb16c04690 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.758830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.759413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.759691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.759963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.760591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.762776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d05666-7eae-4795-99ca-378d268eab67 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.773400 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4034/8140] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:06.777858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.787599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ad97-2b96-47ba-88bd-63082a65e101 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.792208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.792854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.793210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.793487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] PATH_INFO: `/v3/projects/15cc590a447d444f86c525d5e2afd105/users/e5a885ff1fc840fb8e585ea914585338/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.808509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cc590a447d444f86c525d5e2afd105, user_id=e5a885ff1fc840fb8e585ea914585338, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.811871 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:06.825555 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4107/8141] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:06 2022] PUT /identity/v3/projects/15cc590a447d444f86c525d5e2afd105/users/e5a885ff1fc840fb8e585ea914585338/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:06.830591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75227b-4baf-4956-9309-3a720c7ae317 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.830880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75227b-4baf-4956-9309-3a720c7ae317 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.831155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75227b-4baf-4956-9309-3a720c7ae317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.850105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3d75227b-4baf-4956-9309-3a720c7ae317 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:06.851341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d75227b-4baf-4956-9309-3a720c7ae317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:06.909417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4035/8142] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:06.919382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:06.921161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:06.967549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:06.981849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54897a-8615-4929-a51f-1672f73761b5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:06.985519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:06.986121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:06.986403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:06.986705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:06.995737 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:06.997779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:07.007216 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4108/8143] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:10.797269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:10.797758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:10.797865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:10.831975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] MFA Rules not processed for user `e5a885ff1fc840fb8e585ea914585338`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:10.832894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:10.857274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:10.908593 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4036/8144] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:10.925447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:10.929287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:11.021342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc684f-0990-4217-9175-04b61212a0e4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:11.032933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:11.032933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:11.032933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:11.032933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:11.044088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:11.045886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:11.056556 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4109/8145] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:14.739455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:14.753954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f96daf-9fcd-4268-818c-c3a6cf3ba147 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:14.758906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:14.759612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:14.759902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:14.760181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] PATH_INFO: `/v3/users/e5a885ff1fc840fb8e585ea914585338` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:14.761932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a885ff1fc840fb8e585ea914585338)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:14.764096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:14.812221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a885ff1fc840fb8e585ea914585338'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:14.812872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a885ff1fc840fb8e585ea914585338'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:14.819918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a885ff1fc840fb8e585ea914585338'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:14.827879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a885ff1fc840fb8e585ea914585338'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:14.828519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7b0483f3-5603-408d-8363-73b965351d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a885ff1fc840fb8e585ea914585338'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:14.848428 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4037/8146] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:14 2022] DELETE /identity/v3/users/e5a885ff1fc840fb8e585ea914585338 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:15.018648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.029600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649d19-ae43-4d62-8d2f-7a24a351a8fe service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.089857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.090487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.090801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.091080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] PATH_INFO: `/v3/projects/15cc590a447d444f86c525d5e2afd105` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.092200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cc590a447d444f86c525d5e2afd105)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.094114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.141387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cc590a447d444f86c525d5e2afd105', 'request_id': 'req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:15.146986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cc590a447d444f86c525d5e2afd105', 'request_id': 'req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:15.153104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cc590a447d444f86c525d5e2afd105', 'request_id': 'req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:15.165698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] The token cache is being invalidate because project 15cc590a447d444f86c525d5e2afd105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:15.165990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:15.166663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-fdfdd01b-7ccf-4cdf-b7c8-85f3832ac23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:15.168170 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4110/8147] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:27:15 2022] DELETE /identity/v3/projects/15cc590a447d444f86c525d5e2afd105 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:15.174968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.175257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.175538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.195394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:15.196598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:15.255824 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4038/8148] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:15.339240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.339582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.339933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.347954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:15.363444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:15.364649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:15.373220 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4111/8149] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:15.377478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.387302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072dcd2-7bab-4107-b5ac-aca22871ac00 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.392174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.392793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.393077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.393389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.394082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:27:15.394366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.396007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.407482 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4039/8150] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:15.411729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.421772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089bf1c-1722-4760-90a8-425e58075573 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.426603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.427194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.427474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.427742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.428389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.430276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.457975 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4112/8151] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:15.462161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.472256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817058f-a4ce-4913-97ed-7ff0ba904d3b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.492398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:15.531755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.532356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.532672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.532946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.533571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.535554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.559199 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4040/8152] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:15.563537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.574226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdad34-e516-4b18-b5a7-54afc22f70be admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.578870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.579468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.579748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.580022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.580657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.582245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.592069 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4113/8153] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:27:15.596127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.606544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2df51-b038-4be2-8f23-b0d25b08da49 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.611107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.611685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.611972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.612228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.612851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.614427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5824d23-8ffd-4c36-be9d-de649cc0593c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.625099 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4041/8154] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:15.629117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.640075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e49fa-1d99-4c50-81b8-500b55576fe9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.644770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.645519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.645798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.646068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] PATH_INFO: `/v3/projects/434f7c63a1b34202b60a826e930d7d9d/users/7309e1efde7d4f72a36d8213766e67e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.662066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434f7c63a1b34202b60a826e930d7d9d, user_id=7309e1efde7d4f72a36d8213766e67e4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.665197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d480c40-2027-4cd2-b27c-32ce9b841d13 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.679023 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4114/8155] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:15 2022] PUT /identity/v3/projects/434f7c63a1b34202b60a826e930d7d9d/users/7309e1efde7d4f72a36d8213766e67e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:15.689926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.690212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.690485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.718865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] MFA Rules not processed for user `7309e1efde7d4f72a36d8213766e67e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:15.720065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:15.783129 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4042/8156] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:15.794402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.794953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.795294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.814100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] MFA Rules not processed for user `7309e1efde7d4f72a36d8213766e67e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:15.815325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:15.822115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4115/8157] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:15.833775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:15.835149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:15.913152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178f084-e2dc-463a-8130-e866ed4e9401 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:15.919387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:15.919981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:15.920258 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:15.920531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:15.928265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:15.929971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:15.938757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4043/8158] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:16.129528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:16.130972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:16.206516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658836f-74c4-41e5-ac3c-66dda893417f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:16.211571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:16.212180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:16.212459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:16.212732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:16.215167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:16.228883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:16.230665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c09255-0863-4f25-b5b9-845ed03d755f service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:16.243073 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4116/8159] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:25.934792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:26.236367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:55.875809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:55.936857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011fbf2-75f5-4da7-8ce9-dfd9442d0735 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:55.941623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:55.942278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:55.942588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:55.942884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] PATH_INFO: `/v3/users/7309e1efde7d4f72a36d8213766e67e4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:55.944034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7309e1efde7d4f72a36d8213766e67e4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:55.946030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:55.992654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7309e1efde7d4f72a36d8213766e67e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:55.993380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7309e1efde7d4f72a36d8213766e67e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.000556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7309e1efde7d4f72a36d8213766e67e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.009165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7309e1efde7d4f72a36d8213766e67e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.009948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7309e1efde7d4f72a36d8213766e67e4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.027807 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4044/8160] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:55 2022] DELETE /identity/v3/users/7309e1efde7d4f72a36d8213766e67e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:56.033060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.033558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.033688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.052523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:56.053657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:56.111294 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4117/8161] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:56.121318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.122778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:56.182712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae2284-ba5d-45a9-943c-e4398d517ecf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.186385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.187855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.188133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.188401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.197550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.199040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c379e8-0ced-4e4c-b186-fb77a33a8cb3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.209132 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4045/8162] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:27:56.483954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.485412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:56.497156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5861901-d21e-4f4d-af15-78692128bc5f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.502538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.503160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.503438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.503706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] PATH_INFO: `/v3/projects/434f7c63a1b34202b60a826e930d7d9d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.504831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=434f7c63a1b34202b60a826e930d7d9d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.506735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.558242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434f7c63a1b34202b60a826e930d7d9d', 'request_id': 'req-83fb74b4-3b7c-47ad-b747-7390a4eef854'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.564347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434f7c63a1b34202b60a826e930d7d9d', 'request_id': 'req-83fb74b4-3b7c-47ad-b747-7390a4eef854'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.570589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434f7c63a1b34202b60a826e930d7d9d', 'request_id': 'req-83fb74b4-3b7c-47ad-b747-7390a4eef854'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.583375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] The token cache is being invalidate because project 434f7c63a1b34202b60a826e930d7d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:56.583694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.584330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83fb74b4-3b7c-47ad-b747-7390a4eef854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:56.585823 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4118/8163] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:27:56 2022] DELETE /identity/v3/projects/434f7c63a1b34202b60a826e930d7d9d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:56.591505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.591791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.592068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.612141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:56.613334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:56.631287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:56.679468 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4046/8164] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:56.688001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.688287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.688589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.708092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:56.709296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:56.717176 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4119/8165] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:27:56.721439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.731223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4a905-4176-41e0-85e4-a522dc576c66 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.736432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.737021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.737321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.737576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.738208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:27:56.738518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.740143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.752134 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4047/8166] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:56.756298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.766016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e464b-a99e-48b8-97ab-a7b4878de846 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.771227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.771830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.772111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.772405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.773155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.775112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.802715 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4120/8167] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:27:56 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:56.806865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.817794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021703dd-eecd-433f-84cd-a170888604b2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.872159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.872748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.873025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.873292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.873906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.876401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.897317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4048/8168] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:56 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:56.901451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.912647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5216eff-a7e5-47cc-9aee-0de0e50caf03 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.917548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.918149 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.918487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.918800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.919435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.921739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.931589 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4121/8169] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:56.935516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.945770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f85e0-e48f-4b40-8802-951f2a2ead83 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.950468 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.951087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.951368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.951641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.952324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:56.953925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba9b3-71e6-4b5d-9d05-c9814c0056c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:56.964408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4049/8170] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:56.968920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:56.978866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea51d-bf98-4ea6-bd89-3e1dcf8832fb admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:56.984452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:56.985100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:56.985373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:56.985643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] PATH_INFO: `/v3/projects/b4298b8cda9845e98f5ef510b40698de/users/28cc977548fa47728242d5935cc1edd8/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:56.999982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4298b8cda9845e98f5ef510b40698de, user_id=28cc977548fa47728242d5935cc1edd8, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.002650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.017033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4122/8171] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:56 2022] PUT /identity/v3/projects/b4298b8cda9845e98f5ef510b40698de/users/28cc977548fa47728242d5935cc1edd8/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:27:57.027251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.027538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.027814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.056426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] MFA Rules not processed for user `28cc977548fa47728242d5935cc1edd8`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:57.057637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:57.128006 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4050/8172] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:57.132708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.153558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:57.196893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5210881-014e-492b-9a79-eef3117b438d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.201560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.202217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.202595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.202874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.203492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.205396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.231403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4123/8173] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:57.236155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.247025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8dc81-ce68-400b-b525-95626eedb329 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.259286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:57.305222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.305814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.306089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.306358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.306999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.308910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.330135 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4051/8174] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:57.334153 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.345136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000021-0d7f-4876-8b60-d2bb56fa90fc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.349828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.350418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.350738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.351013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.351646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.353244 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.364485 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4124/8175] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:57.368560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.379264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751366e-1a5a-44ba-b742-eb313c6d98a1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.383847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.384490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.384766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.385041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] PATH_INFO: `/v3/projects/45272024792749c8bbf54a873fbcac62/users/c61f4d25ada5435687891dec7a8993e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.400248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45272024792749c8bbf54a873fbcac62, user_id=c61f4d25ada5435687891dec7a8993e9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.402977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.417707 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4052/8176] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:57 2022] PUT /identity/v3/projects/45272024792749c8bbf54a873fbcac62/users/c61f4d25ada5435687891dec7a8993e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:57.422521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.432544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a16819-6062-4006-8956-e655deae6b34 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.488572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.489176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.489458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.489735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.490365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.491992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad10802-e689-434b-a29c-53c2c2bc3f5d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.502378 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4125/8177] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:57.506527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.517024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aeefd-2f47-4636-86df-bc4f62e7c175 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.521664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.522308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.522627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.522904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] PATH_INFO: `/v3/projects/45272024792749c8bbf54a873fbcac62/users/c61f4d25ada5435687891dec7a8993e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.524812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45272024792749c8bbf54a873fbcac62, user_id=c61f4d25ada5435687891dec7a8993e9, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.527491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.537542 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4053/8178] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:27:57 2022] PUT /identity/v3/projects/45272024792749c8bbf54a873fbcac62/users/c61f4d25ada5435687891dec7a8993e9/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:57.549475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.549764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.550044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.581913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] MFA Rules not processed for user `c61f4d25ada5435687891dec7a8993e9`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:27:57.583173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:57.651632 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4126/8179] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:27:57.656750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.667992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a498d1-53d6-4541-a18f-2cb7cc7ba118 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.673487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.674078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.674357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.674676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.675304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.676884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.693317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4054/8180] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/domains => generated 276 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:27:57.697983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.707673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83385fe-23f5-4899-b121-e587cec4f2f3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.719578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:57.765460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.766059 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.766632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.766910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.767604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.769230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.785016 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4127/8181] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/domains => generated 276 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:57.789018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.800106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ac98-ff72-484e-81b2-6f8320d00221 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.855083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.855683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.855968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.856245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.856894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.859197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.874757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4055/8182] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/domains => generated 278 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:57.879275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.889803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c095843-502b-4841-b9ad-8a2471722202 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:57.947449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:57.948049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:57.948328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:57.948600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:57.949281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:57.950934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:57.966657 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4128/8183] 10.222.0.52 () {60 vars in 1150 bytes} [Sat Sep 24 21:27:57 2022] POST /identity/v3/domains => generated 279 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:57.971565 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:57.981873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fd30-37f0-40cd-a680-4e138d05792a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.028585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:58.039621 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.040249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.040529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.040802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/aef3dfa2739d480ca8dbb367191087b6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.051292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=aef3dfa2739d480ca8dbb367191087b6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.052906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.074794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.084764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.123080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.128581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.135492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.147588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project aef3dfa2739d480ca8dbb367191087b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:58.147820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.148463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.156527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.157194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.160500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'aef3dfa2739d480ca8dbb367191087b6', 'request_id': 'req-75c9b3b0-d096-4614-9ea5-2f2e5a138321'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.166014 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4056/8184] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:27:57 2022] DELETE /identity/v3/domains/aef3dfa2739d480ca8dbb367191087b6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:58.171291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.173186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:58.186200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a055dc-5ebf-4350-8c3d-3c33e5744b63 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.242049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.242688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.242974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.243249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.243885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.245509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.261045 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4129/8185] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:27:58 2022] POST /identity/v3/domains => generated 249 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:58.265825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.276193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9b3b0-d096-4614-9ea5-2f2e5a138321 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.331829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.332462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.332743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.333017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/50dbd56a5e744bafa5e774e4510f938e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.344042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=50dbd56a5e744bafa5e774e4510f938e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.345653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.366812 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.367467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.370140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:58.399177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.399762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.402144 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4057/8186] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:27:58 2022] PATCH /identity/v3/domains/50dbd56a5e744bafa5e774e4510f938e => generated 250 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:58.407224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.408917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:58.422052 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5defa-c092-4afc-a6bb-1bce06c35f0c tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.454733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:58.479431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.480066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.480354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.480625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/50dbd56a5e744bafa5e774e4510f938e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.491787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=50dbd56a5e744bafa5e774e4510f938e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.493396 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.515029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.518840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.565135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.570663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.576734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.588525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project 50dbd56a5e744bafa5e774e4510f938e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:58.588808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.589430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.597629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.600877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.601537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '50dbd56a5e744bafa5e774e4510f938e', 'request_id': 'req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.607096 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4130/8187] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:27:58 2022] DELETE /identity/v3/domains/50dbd56a5e744bafa5e774e4510f938e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:58.612687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.622925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8592f7-bdf7-4bfe-8549-25e3f97bd56b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.685063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.685695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.685986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.686264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.686978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.688592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.704296 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4058/8188] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:27:58 2022] POST /identity/v3/domains => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:27:58.708549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.709953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:58.722975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd0087-6074-49d0-b92a-5c4dfaf04f1a tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.780195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.781495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.781846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.782124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/465368e508594ca191bb418c75037a60` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.792494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=465368e508594ca191bb418c75037a60)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.794099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.804336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:58.816151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.816935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.842621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.844024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.846400 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4131/8189] 10.222.0.52 () {60 vars in 1250 bytes} [Sat Sep 24 21:27:58 2022] PATCH /identity/v3/domains/465368e508594ca191bb418c75037a60 => generated 277 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:58.850478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.852231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:58.865410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3daffb-7e08-4966-9c14-9d67de8e0db8 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.923591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.924220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.924503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.924775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/465368e508594ca191bb418c75037a60` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.934984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=465368e508594ca191bb418c75037a60)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.937677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.939248 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4059/8190] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:27:58 2022] GET /identity/v3/domains/465368e508594ca191bb418c75037a60 => generated 277 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:58.943401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:58.953610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9fb9c-0f4b-4ce0-8a98-979b3348ceda tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:58.958300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:58.958948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:58.959234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:58.959507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/465368e508594ca191bb418c75037a60` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:58.960672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=465368e508594ca191bb418c75037a60)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:58.962263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:58.986867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:58.990120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '465368e508594ca191bb418c75037a60'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.037031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.042687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.049711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.062696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project 465368e508594ca191bb418c75037a60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:59.063082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.063740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.071238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.074610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.075282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '465368e508594ca191bb418c75037a60', 'request_id': 'req-53dacaf1-db80-4451-a548-f723130a8d58'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.081156 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4132/8191] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:27:58 2022] DELETE /identity/v3/domains/465368e508594ca191bb418c75037a60 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:59.085247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.086994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:59.099633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d67ddb-2c73-4605-9ba4-9a5af31cdc2b tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.155913 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.156665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.156947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.157225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.157831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.159461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.172535 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4060/8192] 10.222.0.52 () {58 vars in 1128 bytes} [Sat Sep 24 21:27:59 2022] GET /identity/v3/domains => generated 1348 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:27:59.176825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.187022 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dacaf1-db80-4451-a548-f723130a8d58 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.191897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.192448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.192728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.193001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/465368e508594ca191bb418c75037a60` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.199826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find domain: 465368e508594ca191bb418c75037a60.: keystone.exception.DomainNotFound: Could not find domain: 465368e508594ca191bb418c75037a60. Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 465368e508594ca191bb418c75037a60. Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:27:59.205109 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:27:59.206692 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 465368e508594ca191bb418c75037a60. Sep 24 21:27:59.208104 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.209533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4133/8193] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:27:59 2022] PATCH /identity/v3/domains/465368e508594ca191bb418c75037a60 => generated 112 bytes in 30 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:27:59.210993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.221503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa2944-dc49-4808-953a-abe2089ba002 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.226187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.226807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.227090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.227363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.228019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.229597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.233154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:59.246862 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4061/8194] 10.222.0.52 () {60 vars in 1149 bytes} [Sat Sep 24 21:27:59 2022] POST /identity/v3/domains => generated 276 bytes in 36 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:27:59.251103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.261739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ecec-3502-46d3-bc94-718f4e6433eb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.317892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.318512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.318810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.319085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.319720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.321638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.326114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:59.355622 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4134/8195] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:27:59 2022] POST /identity/v3/users => generated 348 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:59.360547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.371032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc48ea26-50c0-4342-a1b0-70debd529cba tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.375908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.376513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.376782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.377074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/groups` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.378029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.380011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.394719 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4062/8196] 10.222.0.52 () {60 vars in 1147 bytes} [Sat Sep 24 21:27:59 2022] POST /identity/v3/groups => generated 325 bytes in 35 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:27:59.398695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.409547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4ff03-b548-44d6-8dd5-af6b8a021f07 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.414290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.414944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.415232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.415534 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/9a9cffd4df5043a99c02028e9b15fb94` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.416760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=9a9cffd4df5043a99c02028e9b15fb94)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.418348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.440420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.441128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.465405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.465968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.469078 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4135/8197] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:27:59 2022] PATCH /identity/v3/domains/9a9cffd4df5043a99c02028e9b15fb94 => generated 277 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:27:59.472881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.474629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:59.488251 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da925ee-5992-454f-9c8f-2e52223775d6 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.543863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.544486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.544768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.545046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/9a9cffd4df5043a99c02028e9b15fb94` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.555584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=9a9cffd4df5043a99c02028e9b15fb94)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.557599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.580415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.690039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ba464d79de44e68cd53c73109213ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.690769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ba464d79de44e68cd53c73109213ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.692382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:27:59.698408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ba464d79de44e68cd53c73109213ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.707197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ba464d79de44e68cd53c73109213ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.707944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ba464d79de44e68cd53c73109213ed'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.724752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.766392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.772377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.779853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.791505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project 9a9cffd4df5043a99c02028e9b15fb94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:27:59.791792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.792425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.799974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.800616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.803831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '9a9cffd4df5043a99c02028e9b15fb94', 'request_id': 'req-f3befd62-9748-4914-966f-91f6643d93ce'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:27:59.809284 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4063/8198] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:27:59 2022] DELETE /identity/v3/domains/9a9cffd4df5043a99c02028e9b15fb94 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:27:59.813403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.823439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891153f5-50d1-4e9f-bdae-673b12384009 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.884678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.885323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.885607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.885885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/9a9cffd4df5043a99c02028e9b15fb94` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.893849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=9a9cffd4df5043a99c02028e9b15fb94)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.895729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94.: keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:27:59.907487 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:27:59.908922 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.910346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4136/8199] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:27:59 2022] GET /identity/v3/domains/9a9cffd4df5043a99c02028e9b15fb94 => generated 112 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:27:59.912800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.914295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:27:59.927262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3befd62-9748-4914-966f-91f6643d93ce tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.931925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.932537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.932817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.933093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/users/34ba464d79de44e68cd53c73109213ed` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.941970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:get_user(user_id=34ba464d79de44e68cd53c73109213ed)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.944024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find user: 34ba464d79de44e68cd53c73109213ed.: keystone.exception.UserNotFound: Could not find user: 34ba464d79de44e68cd53c73109213ed. Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 21:27:59.956102 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:27:59.957428 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 34ba464d79de44e68cd53c73109213ed. Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:27:59.958930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4064/8200] 10.222.0.52 () {58 vars in 1221 bytes} [Sat Sep 24 21:27:59 2022] GET /identity/v3/users/34ba464d79de44e68cd53c73109213ed => generated 110 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:27:59.961254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:27:59.971483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b5ba6-3212-473e-9561-878dc499d3a7 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:27:59.976229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:27:59.976788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:27:59.977070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:27:59.977344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/groups/eccb60b2b5a646ae81768c738971892b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:27:59.981800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:get_group(group_id=eccb60b2b5a646ae81768c738971892b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:27:59.983692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find group: eccb60b2b5a646ae81768c738971892b.: keystone.exception.GroupNotFound: Could not find group: eccb60b2b5a646ae81768c738971892b. Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_group(group_id) Sep 24 21:27:59.995114 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:27:59.996440 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: eccb60b2b5a646ae81768c738971892b. Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:27:59.997831 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4137/8201] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:27:59 2022] GET /identity/v3/groups/eccb60b2b5a646ae81768c738971892b => generated 111 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:28:00.000646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.010694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37d52-0847-458a-abc2-4772857a25f1 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.016377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.017243 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.017522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.017792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/groups/eccb60b2b5a646ae81768c738971892b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.022853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find group: eccb60b2b5a646ae81768c738971892b.: keystone.exception.GroupNotFound: Could not find group: eccb60b2b5a646ae81768c738971892b. Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete Sep 24 21:28:00.027888 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:28:00.029093 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: eccb60b2b5a646ae81768c738971892b. Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.030267 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4065/8202] 10.222.0.52 () {58 vars in 1227 bytes} [Sat Sep 24 21:27:59 2022] DELETE /identity/v3/groups/eccb60b2b5a646ae81768c738971892b => generated 111 bytes in 29 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 24 21:28:00.033018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.043498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf81ebb-b80c-48a1-ace3-e6dee2711c04 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.048222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.048826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.049112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.049391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/users/34ba464d79de44e68cd53c73109213ed` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.056665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_user(user_id=34ba464d79de44e68cd53c73109213ed)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.058547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find user: 34ba464d79de44e68cd53c73109213ed.: keystone.exception.UserNotFound: Could not find user: 34ba464d79de44e68cd53c73109213ed. Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete Sep 24 21:28:00.070417 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:28:00.071708 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application with Lock( Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 34ba464d79de44e68cd53c73109213ed. Sep 24 21:28:00.073161 ncdv-master-1 devstack@keystone.service[99132]: ERROR keystone.server.flask.application Sep 24 21:28:00.074743 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4138/8203] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:00 2022] DELETE /identity/v3/users/34ba464d79de44e68cd53c73109213ed => generated 110 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:28:00.075914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.086750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08dc6-b734-492c-9469-03630dcb3917 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.091277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.091878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.092159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.092430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/9a9cffd4df5043a99c02028e9b15fb94` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.099980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94.: keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:28:00.105054 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:00.106555 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.107894 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.109309 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4066/8204] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:28:00 2022] PATCH /identity/v3/domains/9a9cffd4df5043a99c02028e9b15fb94 => generated 112 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:28:00.112005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.122577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39017b-d4a6-4cd2-968b-40eabb745cc9 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.128012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.128592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.128887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.129138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.129735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.131357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.144791 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4139/8205] 10.222.0.52 () {58 vars in 1128 bytes} [Sat Sep 24 21:28:00 2022] GET /identity/v3/domains => generated 1348 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:28:00.149674 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.159773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf0990-ce2a-44c5-b5e6-50c788164c52 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.164376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.164961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.165245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.165520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.166152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:00.166452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.168088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.186259 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4067/8206] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:00 2022] GET /identity/v3/domains?enabled=True => generated 1094 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:00.192072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.202221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabb426-1c0a-4d53-a9ab-89b104076263 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.207845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.208433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.208716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.208992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.209624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-1902738605) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:00.209931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.211549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.223414 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4140/8207] 10.222.0.52 () {58 vars in 1199 bytes} [Sat Sep 24 21:28:00 2022] GET /identity/v3/domains?name=tempest-test_domain-1902738605 => generated 408 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:00.228583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.239017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de05a7-871c-488f-a65a-2c2bd1c338ac tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.243620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.244233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.244516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.244788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/9a9cffd4df5043a99c02028e9b15fb94` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.252198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94.: keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:28:00.257139 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:00.258545 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 9a9cffd4df5043a99c02028e9b15fb94. Sep 24 21:28:00.259658 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:00.261229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4068/8208] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:28:00 2022] PATCH /identity/v3/domains/9a9cffd4df5043a99c02028e9b15fb94 => generated 112 bytes in 30 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:28:00.262021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.272781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cf4b8-1cbd-4781-9721-d7086ab60144 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.277339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.277943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.278225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.278524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/7dc2b62b06734d84b5824ed59f69af98` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.289811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=7dc2b62b06734d84b5824ed59f69af98)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.291484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.324731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:UPDATE=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:00.336971 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4141/8209] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:28:00 2022] PATCH /identity/v3/domains/7dc2b62b06734d84b5824ed59f69af98 => generated 278 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:28:00.341069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.353236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd87c4-9a03-4a4f-a14b-6728434cd444 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.357942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.359197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.359487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.359901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/7dc2b62b06734d84b5824ed59f69af98` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.372519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=7dc2b62b06734d84b5824ed59f69af98)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.374162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.397555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.406733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.407908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=2,keystone:DELETE=14,keystone:SELECT=84 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:00.449800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.455560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.462691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.474210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project 7dc2b62b06734d84b5824ed59f69af98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:00.474512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.475150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.482453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.483151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.487180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '7dc2b62b06734d84b5824ed59f69af98', 'request_id': 'req-a661871a-48c5-4b57-bf2b-16d5f0456a55'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.492674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4069/8210] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:28:00 2022] DELETE /identity/v3/domains/7dc2b62b06734d84b5824ed59f69af98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:00.496886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.498724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:00.511814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2409e7-4b46-4f6c-a167-139162af9360 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.569049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.569686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.569985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.570257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/6e54ce68c72540378bcc11124197bdec` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.581061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=6e54ce68c72540378bcc11124197bdec)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.582701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.603561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.604216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.629208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.629850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.635322 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4142/8211] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:28:00 2022] PATCH /identity/v3/domains/6e54ce68c72540378bcc11124197bdec => generated 277 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:00.636490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.646872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661871a-48c5-4b57-bf2b-16d5f0456a55 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.703660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.704288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.704569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.704844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/6e54ce68c72540378bcc11124197bdec` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.715521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=6e54ce68c72540378bcc11124197bdec)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.717143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.740419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.750945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '6e54ce68c72540378bcc11124197bdec'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.788958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:00.793132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.800273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.807809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.819349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project 6e54ce68c72540378bcc11124197bdec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:00.819636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.820274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.827619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.828324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.831588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '6e54ce68c72540378bcc11124197bdec', 'request_id': 'req-a7108dfd-617b-45bb-b962-cbd2653949ee'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.837011 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4070/8212] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:28:00 2022] DELETE /identity/v3/domains/6e54ce68c72540378bcc11124197bdec => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:00.840961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.842383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:00.855855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626475cb-dc24-4363-bbe7-43d822dd4f16 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:00.913025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:00.913660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:00.913942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:00.914216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/e9e510b4ecf2409d8b702eb134fa825e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:00.925675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=e9e510b4ecf2409d8b702eb134fa825e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:00.927313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:00.947697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.948363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.973110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.973739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:00.976277 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4143/8213] 10.222.0.52 () {60 vars in 1249 bytes} [Sat Sep 24 21:28:00 2022] PATCH /identity/v3/domains/e9e510b4ecf2409d8b702eb134fa825e => generated 277 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:28:00.980384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:00.981792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:00.995133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108dfd-617b-45bb-b962-cbd2653949ee tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:01.052984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c61f4d25ada5435687891dec7a8993e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45272024792749c8bbf54a873fbcac62', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:01.053619 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:01.054281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:01.054651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] PATH_INFO: `/v3/domains/e9e510b4ecf2409d8b702eb134fa825e` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:01.065755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=e9e510b4ecf2409d8b702eb134fa825e)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:01.067729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:01.089772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.100145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.138435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.144866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.152157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.164263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] The token cache is being invalidate because project e9e510b4ecf2409d8b702eb134fa825e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:01.164554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.165199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.170939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:01.173996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.175144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.179318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-78b8d84d-f658-47c0-a12a-a28d6377ffdb tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'e9e510b4ecf2409d8b702eb134fa825e', 'request_id': 'req-78b8d84d-f658-47c0-a12a-a28d6377ffdb'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.185734 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4071/8214] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:28:00 2022] DELETE /identity/v3/domains/e9e510b4ecf2409d8b702eb134fa825e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:01.189813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:01.191363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:01.200972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:01.257099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a1a77-efaf-41a3-86a2-d80468a0d9e4 tempest-DomainsTestJSON-951997933 tempest-DomainsTestJSON-951997933-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:01.314733 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:01.315395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:01.315680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:01.315952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] PATH_INFO: `/v3/users/28cc977548fa47728242d5935cc1edd8` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:01.317229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cc977548fa47728242d5935cc1edd8)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:01.319236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:01.365139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cc977548fa47728242d5935cc1edd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.365801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cc977548fa47728242d5935cc1edd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.373328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cc977548fa47728242d5935cc1edd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.386146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cc977548fa47728242d5935cc1edd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.386812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cc977548fa47728242d5935cc1edd8'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.399670 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4144/8215] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:01 2022] DELETE /identity/v3/users/28cc977548fa47728242d5935cc1edd8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:01.404098 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:01.404400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:01.404655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:01.423681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:01.424884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:01.485395 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4072/8216] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:01.495044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:01.496483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:01.533892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:01.560434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb38f4-4957-42ed-97ea-d7b388b27106 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:01.564990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:01.565594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:01.565876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:01.566161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:01.575227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:01.576661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:01.587013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4145/8217] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:01.862525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:01.863974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:01.880712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198bdb27-668f-4594-8050-16ced8c88837 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:01.887651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:01.888196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:01.888475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:01.888747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] PATH_INFO: `/v3/projects/b4298b8cda9845e98f5ef510b40698de` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:01.890450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4298b8cda9845e98f5ef510b40698de)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:01.892615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:01.939184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4298b8cda9845e98f5ef510b40698de', 'request_id': 'req-c85893bf-8236-4770-99b5-e80b39ed9a89'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.945562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4298b8cda9845e98f5ef510b40698de', 'request_id': 'req-c85893bf-8236-4770-99b5-e80b39ed9a89'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.953581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4298b8cda9845e98f5ef510b40698de', 'request_id': 'req-c85893bf-8236-4770-99b5-e80b39ed9a89'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.964750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] The token cache is being invalidate because project b4298b8cda9845e98f5ef510b40698de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:01.965039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.965688 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:01.967192 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4073/8218] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:01 2022] DELETE /identity/v3/projects/b4298b8cda9845e98f5ef510b40698de => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:01.972062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:01.982206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368e645-3e8a-4b82-ad33-f01fb2409b19 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.038335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.038991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.039268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.039537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] PATH_INFO: `/v3/users/c61f4d25ada5435687891dec7a8993e9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.040640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61f4d25ada5435687891dec7a8993e9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.042566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.087170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61f4d25ada5435687891dec7a8993e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.087809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61f4d25ada5435687891dec7a8993e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.096130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61f4d25ada5435687891dec7a8993e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.108074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61f4d25ada5435687891dec7a8993e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.108746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cbeea3a8-1747-4565-81e6-32f2c55b1fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61f4d25ada5435687891dec7a8993e9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.120639 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4146/8219] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:01 2022] DELETE /identity/v3/users/c61f4d25ada5435687891dec7a8993e9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:02.392477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.393922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:02.406973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85893bf-8236-4770-99b5-e80b39ed9a89 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.468803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.469444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.469724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.469994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] PATH_INFO: `/v3/projects/45272024792749c8bbf54a873fbcac62` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.471146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45272024792749c8bbf54a873fbcac62)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.473503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.490963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:02.523051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45272024792749c8bbf54a873fbcac62', 'request_id': 'req-ce615251-3b0d-40e1-8ec0-a471ca041985'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.528558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45272024792749c8bbf54a873fbcac62', 'request_id': 'req-ce615251-3b0d-40e1-8ec0-a471ca041985'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.536527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45272024792749c8bbf54a873fbcac62', 'request_id': 'req-ce615251-3b0d-40e1-8ec0-a471ca041985'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.548363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] The token cache is being invalidate because project 45272024792749c8bbf54a873fbcac62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:02.548649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.549306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ce615251-3b0d-40e1-8ec0-a471ca041985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:02.550827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4074/8220] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:02 2022] DELETE /identity/v3/projects/45272024792749c8bbf54a873fbcac62 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:02.555653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.556146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.556255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.576619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:02.577912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:02.627160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:02.645393 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4147/8221] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:02.649833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.650118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.650393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.670547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:02.672445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:02.680280 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4075/8222] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:02.684406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.694728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6d7d9-7cf3-498b-bf79-3402369adb48 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.699259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.699829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.700101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.700367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.700991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:02.701285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.702954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.714487 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4148/8223] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:28:02.718917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.728165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142d88f-206b-46f8-aa1b-82ad7aeb6cef None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.732872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.733460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.734391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.734699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.735428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.737394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.765133 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4076/8224] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:02 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:02.769685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.780072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b18cbb-880a-40ad-893f-4ebcc6dfd9f8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.836891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.837518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.837800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.838072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.838742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.840691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.862248 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4149/8225] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:02 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:02.866331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.877682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a39cc5-6cbf-4f43-8b15-9be90d7dbd95 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.882331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.882963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.883244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.883539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.884166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.885819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.896203 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4077/8226] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:02.900278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.911231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250abcc-8c8d-47c4-b291-b8f3a567671b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.915903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.916485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.916760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.917029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.917662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.919287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153a6e-e9ac-472b-869b-58e0443884c1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.930933 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4150/8227] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:02.935045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:02.945056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846f49-cb33-46c2-a462-deaa48c1da32 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:02.949647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:02.950286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.950611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.950886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] PATH_INFO: `/v3/projects/d0fe7f400df94b1eb3195cc2ab205005/users/163986694d3a4352a8bb22b153ca711f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:02.965337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fe7f400df94b1eb3195cc2ab205005, user_id=163986694d3a4352a8bb22b153ca711f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:02.968568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:02.981317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4078/8228] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:02 2022] PUT /identity/v3/projects/d0fe7f400df94b1eb3195cc2ab205005/users/163986694d3a4352a8bb22b153ca711f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:02.993419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:02.993704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:02.993974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.022061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] MFA Rules not processed for user `163986694d3a4352a8bb22b153ca711f`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:03.023315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:03.087390 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4151/8229] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:03.091851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.152976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e79fb-f450-42a7-aa41-3029af0ec8f4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.158377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.158993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.159270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.159542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.160149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.162043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.189552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4079/8230] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:03.193538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.204469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33038462-4dea-49be-a2b6-f1288ded1b68 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.251634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:03.263023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.263571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.263999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.264295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.265436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.267692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.291343 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4152/8231] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:28:03.293608 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.304262 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dceb185-7757-4e68-879b-daf7ada0f908 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.308934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.309527 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.309803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.310075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.310744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.312634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.323250 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4080/8232] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:03.327296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.337979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10df5c-c7e8-4cdf-8b55-66c620261313 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.342810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.344083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.344367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.344638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] PATH_INFO: `/v3/projects/d2dda331f13c4cf686eb163376e77e78/users/fd66e391ea6d45b497c13bddee489d46/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.361171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2dda331f13c4cf686eb163376e77e78, user_id=fd66e391ea6d45b497c13bddee489d46, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.363930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.378015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4153/8233] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:03 2022] PUT /identity/v3/projects/d2dda331f13c4cf686eb163376e77e78/users/fd66e391ea6d45b497c13bddee489d46/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:03.382232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.392957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4bce3-3bfa-4119-b643-b7c9081e54b7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.402000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:03.451138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.451667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.451944 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.452214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.452843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.454444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c5aab-e392-407c-9079-07383014980e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.464015 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4081/8234] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:03.468122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.478750 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6340dba-7b5d-41fd-b23f-d9a51e1f81c4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.483370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.484644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.485120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.485389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] PATH_INFO: `/v3/projects/d2dda331f13c4cf686eb163376e77e78/users/fd66e391ea6d45b497c13bddee489d46/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.487332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2dda331f13c4cf686eb163376e77e78, user_id=fd66e391ea6d45b497c13bddee489d46, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.489960 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.501240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4154/8235] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:03 2022] PUT /identity/v3/projects/d2dda331f13c4cf686eb163376e77e78/users/fd66e391ea6d45b497c13bddee489d46/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:03.511325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.511638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.511914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.539650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] MFA Rules not processed for user `fd66e391ea6d45b497c13bddee489d46`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:03.540857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:03.596781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.598660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:03.616267 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4082/8236] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:03.621393 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.633738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6d82-04bf-4cdb-95b1-ab44191e5f3a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.638935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.639557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.639841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.640121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.642568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.642884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.662063 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4083/8237] 10.222.0.52 () {60 vars in 1152 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/services => generated 239 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:28:03.663169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.684904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136305-7283-4209-bb75-c142063d9bac tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.689876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.690631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.690915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.691191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/3b22d9ec0abf4939b02c8103c815edbe` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.691772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:delete_service(service_id=3b22d9ec0abf4939b02c8103c815edbe)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.693398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.699382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7b943-1836-4c29-be10-12fc783e93d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.715082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.715588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.715869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.716003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4084/8238] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:03 2022] DELETE /identity/v3/services/3b22d9ec0abf4939b02c8103c815edbe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:03.716126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.718013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:03.721309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.733379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.735824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.742264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd20a-8fd7-4079-ad63-8aa44e38cc8c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.749049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.749828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.750103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.750242 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4155/8239] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:03 2022] GET /identity/v3/auth/tokens => generated 3440 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:28:03.750371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.751045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.752638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.769988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4085/8240] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/services => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:28:03.774347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.785762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae970-4b16-44a8-b51e-ad007ac6ea6b service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.802779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.803413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `PATCH` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.803700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.803974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/0a3dd34c3afc44448f39b15b5e27f06d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.804630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:update_service(service_id=0a3dd34c3afc44448f39b15b5e27f06d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.805645 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:28:03.805645 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:28:03.805645 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:28:03.806551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.828567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4156/8241] 10.222.0.52 () {60 vars in 1252 bytes} [Sat Sep 24 21:28:03 2022] PATCH /identity/v3/services/0a3dd34c3afc44448f39b15b5e27f06d => generated 283 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:03.829745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.840845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55064e-9985-4080-96df-8e12b1195698 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.857700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.858327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.858653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.859060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/0a3dd34c3afc44448f39b15b5e27f06d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.859917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:get_service(service_id=0a3dd34c3afc44448f39b15b5e27f06d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.860943 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:28:03.860943 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:28:03.860943 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:28:03.861828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.867511 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4086/8242] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:28:03 2022] GET /identity/v3/services/0a3dd34c3afc44448f39b15b5e27f06d => generated 283 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:03.871405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.882538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84b8da-09fc-4c1d-8c77-7d7269a5f2e9 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.887261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.887878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.888162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.888437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/0a3dd34c3afc44448f39b15b5e27f06d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.889013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:delete_service(service_id=0a3dd34c3afc44448f39b15b5e27f06d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.891270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.913171 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4157/8243] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:03 2022] DELETE /identity/v3/services/0a3dd34c3afc44448f39b15b5e27f06d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:03.917491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.928011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ee2f0-f2ba-45ad-b311-72ad008706ff tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.934007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:03.945923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.946895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.947175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.947445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/0a3dd34c3afc44448f39b15b5e27f06d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.948016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:get_service(service_id=0a3dd34c3afc44448f39b15b5e27f06d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.949607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Could not find service: 0a3dd34c3afc44448f39b15b5e27f06d.: keystone.exception.ServiceNotFound: Could not find service: 0a3dd34c3afc44448f39b15b5e27f06d. Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 0a3dd34c3afc44448f39b15b5e27f06d. Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 21:28:03.959431 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 21:28:03.960833 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application with Lock( Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self._enter() Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application return self.creator() Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application created_value = creator( Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: 0a3dd34c3afc44448f39b15b5e27f06d. Sep 24 21:28:03.962206 ncdv-master-1 devstack@keystone.service[99133]: ERROR keystone.server.flask.application Sep 24 21:28:03.963563 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4087/8244] 10.222.0.52 () {58 vars in 1230 bytes} [Sat Sep 24 21:28:03 2022] GET /identity/v3/services/0a3dd34c3afc44448f39b15b5e27f06d => generated 113 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 24 21:28:03.965629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:03.976793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65873077-958b-418b-809d-a45f54b4e462 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:03.981387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:03.981975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:03.982255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:03.982560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:03.983215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:03.985351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:03.999031 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4158/8245] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:03 2022] POST /identity/v3/services => generated 273 bytes in 34 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:04.002921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.013391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a697169-a076-47c6-93d0-c03115361c5c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.028939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.029528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.029808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.030107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.031471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.033062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.047345 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4088/8246] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:04 2022] POST /identity/v3/services => generated 275 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:04.051297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.061583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9216527-6e33-4c2f-ad55-1201d9ee0318 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.072175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:04.078127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.078807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.079250 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.079869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.080508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.082205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.098186 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4159/8247] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:04 2022] POST /identity/v3/services => generated 274 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:04.102324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.112798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf51697-b96d-4d60-b368-25b77192878d tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.127450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.128027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.128307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.128582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.129176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.130185 ncdv-master-1 devstack@keystone.service[99133]: WARNING py.warnings [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:28:04.130185 ncdv-master-1 devstack@keystone.service[99133]: warnings.warn(msg) Sep 24 21:28:04.130185 ncdv-master-1 devstack@keystone.service[99133]: Sep 24 21:28:04.131112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.138571 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4089/8248] 10.222.0.52 () {58 vars in 1131 bytes} [Sat Sep 24 21:28:04 2022] GET /identity/v3/services => generated 2977 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:04.143515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.153735 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7f57-376f-4b6b-9f5b-05db7229ab65 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.159103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.159687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.159968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.160241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.160859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-508015259) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:04.161157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.162149 ncdv-master-1 devstack@keystone.service[99132]: WARNING py.warnings [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 21:28:04.162149 ncdv-master-1 devstack@keystone.service[99132]: warnings.warn(msg) Sep 24 21:28:04.162149 ncdv-master-1 devstack@keystone.service[99132]: Sep 24 21:28:04.163054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.167851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4160/8249] 10.222.0.52 () {58 vars in 1220 bytes} [Sat Sep 24 21:28:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-508015259 => generated 415 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:04.172017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.182272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21aee7-8be0-4b2c-b67c-db1ffa9f4b35 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.187685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.188271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.188552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.188829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.189461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1892932188) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:04.189833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.191449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.197071 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4090/8250] 10.222.0.52 () {58 vars in 1222 bytes} [Sat Sep 24 21:28:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1892932188 => generated 418 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:04.201070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.212112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62c733-508d-4166-89bc-a9fe468030b6 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.217649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.219136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.219428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.219777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.220397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-351568188) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:04.220696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.222601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.227567 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4161/8251] 10.222.0.52 () {58 vars in 1220 bytes} [Sat Sep 24 21:28:04 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-351568188 => generated 416 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:04.231789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.242443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007df52a-8943-4291-ab8a-0454ad1baac1 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.247174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.247788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.248062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.248330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/b412bd12d5b14f8fb2ca3360f33245de` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.248893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:delete_service(service_id=b412bd12d5b14f8fb2ca3360f33245de)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.250582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.274633 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4091/8252] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:04 2022] DELETE /identity/v3/services/b412bd12d5b14f8fb2ca3360f33245de => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:04.278651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.289660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc710e0f-b2e3-4ccc-821a-6f2981a405c4 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.306258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.306936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.307211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.307483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/f64101ee330b48498375716a6dafdca3` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.308056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:delete_service(service_id=f64101ee330b48498375716a6dafdca3)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.309668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.331305 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4162/8253] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:04 2022] DELETE /identity/v3/services/f64101ee330b48498375716a6dafdca3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:04.335406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.346137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aae0d9-9155-4e2f-b093-675b09fb5f7f tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.361446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd66e391ea6d45b497c13bddee489d46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd2dda331f13c4cf686eb163376e77e78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.362077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.362355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.362680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] PATH_INFO: `/v3/services/9333dd603fd04e92b6f32e226c448378` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.363237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorizing `identity:delete_service(service_id=9333dd603fd04e92b6f32e226c448378)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.364839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a82aff-2e87-45f2-a870-304840268e3c tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.387275 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4092/8254] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:04 2022] DELETE /identity/v3/services/9333dd603fd04e92b6f32e226c448378 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:04.393090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.463896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7240f-7f36-4873-8356-271a1c5f6e71 tempest-ServicesTestJSON-1103140551 tempest-ServicesTestJSON-1103140551-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.478399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.478990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.479276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.479545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] PATH_INFO: `/v3/users/163986694d3a4352a8bb22b153ca711f` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.480705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=163986694d3a4352a8bb22b153ca711f)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.482645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.528498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163986694d3a4352a8bb22b153ca711f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:04.529166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163986694d3a4352a8bb22b153ca711f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:04.537486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163986694d3a4352a8bb22b153ca711f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:04.549835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163986694d3a4352a8bb22b153ca711f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:04.550479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163986694d3a4352a8bb22b153ca711f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:04.558825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:04.563770 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4163/8255] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:04 2022] DELETE /identity/v3/users/163986694d3a4352a8bb22b153ca711f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:04.568558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.569070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.569196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.588046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:04.589249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:04.649668 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4093/8256] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:04.660036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:04.661428 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:04.731080 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aaa64-3ad5-4381-bdcc-f1d14d51f524 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:04.735421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:04.736016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:04.736294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:04.736566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:04.745570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:04.747031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:04.756686 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4164/8257] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:05.022771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.024208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:05.038225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40bccf-b6c1-4cd8-9988-5470b137c1c2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:05.043106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:05.043745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.044026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.044300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] PATH_INFO: `/v3/projects/d0fe7f400df94b1eb3195cc2ab205005` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.045427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0fe7f400df94b1eb3195cc2ab205005)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:05.047631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:05.089198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:05.096670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0fe7f400df94b1eb3195cc2ab205005', 'request_id': 'req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.102276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0fe7f400df94b1eb3195cc2ab205005', 'request_id': 'req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.109681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0fe7f400df94b1eb3195cc2ab205005', 'request_id': 'req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.120726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] The token cache is being invalidate because project d0fe7f400df94b1eb3195cc2ab205005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:05.121014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.121644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.123141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4094/8258] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:05 2022] DELETE /identity/v3/projects/d0fe7f400df94b1eb3195cc2ab205005 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:05.127877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.137605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5c887-374c-4bc8-af4b-30b9e6656fca service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:05.194275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:05.194990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.195268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.195539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] PATH_INFO: `/v3/users/fd66e391ea6d45b497c13bddee489d46` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.196642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd66e391ea6d45b497c13bddee489d46)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:05.198569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:05.243763 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd66e391ea6d45b497c13bddee489d46'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.244411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd66e391ea6d45b497c13bddee489d46'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.252110 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd66e391ea6d45b497c13bddee489d46'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.263818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd66e391ea6d45b497c13bddee489d46'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.264500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-98daa958-aa05-4c7f-a397-20841318c078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd66e391ea6d45b497c13bddee489d46'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.277643 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4165/8259] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:05 2022] DELETE /identity/v3/users/fd66e391ea6d45b497c13bddee489d46 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:05.564440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.565902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:05.579082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bbbee-e92a-4ba4-8dc5-977e7498f1c4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:05.638921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:05.639550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.639826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.640714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] PATH_INFO: `/v3/projects/d2dda331f13c4cf686eb163376e77e78` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.642358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2dda331f13c4cf686eb163376e77e78)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:05.645204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:05.696839 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2dda331f13c4cf686eb163376e77e78', 'request_id': 'req-68f5c7fc-b614-4590-901a-361d3b7ddc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.702223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2dda331f13c4cf686eb163376e77e78', 'request_id': 'req-68f5c7fc-b614-4590-901a-361d3b7ddc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.710167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2dda331f13c4cf686eb163376e77e78', 'request_id': 'req-68f5c7fc-b614-4590-901a-361d3b7ddc8f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.722081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] The token cache is being invalidate because project d2dda331f13c4cf686eb163376e77e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:05.722365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.723027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-68f5c7fc-b614-4590-901a-361d3b7ddc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:05.724532 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4095/8260] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:05 2022] DELETE /identity/v3/projects/d2dda331f13c4cf686eb163376e77e78 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:05.731123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.731610 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.731726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.751753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:05.752962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:05.761321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:05.817019 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4166/8261] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:05.824853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.825122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.825394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.846022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:05.847259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:05.855101 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4096/8262] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:05.859435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.869265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c02b6-bbd5-40ae-8fa0-5c70fed042b1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:05.873935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:05.875233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.875661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.876093 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.876732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:05.877032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:05.878656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:05.889982 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4167/8263] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:28:05.895188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.904717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e6d50-6e27-45b0-ab19-7e59a2967410 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:05.909623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:05.910228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:05.910544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:05.910848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:05.911478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:05.913387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:05.940999 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4097/8264] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:05 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:05.945139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:05.956051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967a4ac-0f3b-4bf7-9213-72a37699fc1b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.011701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.012299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.012580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.012854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.013471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.016090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.037338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4168/8265] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:05 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:28:06.041461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.051791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277024e-1969-4b6c-8f28-4819922855c2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.056825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.057422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.057702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.058004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.058756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.060356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.071321 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4098/8266] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:06.075124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.086082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cfbe6-a9d0-4f0b-b64b-0e7f7954e39e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.090791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.091380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.091661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.091937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.092583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.094223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f14c1-33cb-494f-8a8a-d70bb10c6909 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.104921 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4169/8267] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:28:06.109225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.119046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127434be-1af2-468c-9cbd-6a9b4de0f518 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.123590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.124222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.125000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.125278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] PATH_INFO: `/v3/projects/d3d8c2e4e34e4105bf5747711fcedf6b/users/ed1eec0df5da4e5281e630005d1ca682/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.139412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d8c2e4e34e4105bf5747711fcedf6b, user_id=ed1eec0df5da4e5281e630005d1ca682, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.142941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.145911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:06.158140 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4099/8268] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:06 2022] PUT /identity/v3/projects/d3d8c2e4e34e4105bf5747711fcedf6b/users/ed1eec0df5da4e5281e630005d1ca682/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:06.169595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.169910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.170185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.197933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] MFA Rules not processed for user `ed1eec0df5da4e5281e630005d1ca682`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:06.199199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:06.261998 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4170/8269] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:06.266435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.331049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04081cc2-d50a-4e3c-a6f9-f910cd65d98c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.335666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.336263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.336541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.336813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.337475 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.339416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.366053 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4100/8270] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:06 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:06.370246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.382306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2ce8-ea18-43b2-9054-a8ac62d096b1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.387021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:06.441334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.441938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.442220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.442512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.443163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.445078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.470608 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4171/8271] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:06 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:06.471978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.482213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e19b-ac43-4634-ac3f-53633ab425a7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.488268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.488887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.489163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.489434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.490060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.491711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.502974 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4101/8272] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:06.506961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.517543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c97cd-c842-4ba4-9789-3ab4f916a891 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.522207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.522817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.523099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.523368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.523999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.525605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5370bb0-1400-462c-a88a-fb69e9fd9840 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.539381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4172/8273] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:06.540768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.551744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98f36c-6860-43d1-9511-0e205980dcdd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.556416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.557056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.557336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.557610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] PATH_INFO: `/v3/projects/e14176f90d294f8da9d1bfb0921afe01/users/99c58044cc564946b9cf8d65ddb89f46/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.573910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14176f90d294f8da9d1bfb0921afe01, user_id=99c58044cc564946b9cf8d65ddb89f46, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.576606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.589662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4102/8274] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:06 2022] PUT /identity/v3/projects/e14176f90d294f8da9d1bfb0921afe01/users/99c58044cc564946b9cf8d65ddb89f46/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:06.599939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.600226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.600547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.633293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] MFA Rules not processed for user `99c58044cc564946b9cf8d65ddb89f46`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:06.634549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:06.702521 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4173/8275] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:06.714336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:06.716002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:06.798904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6ed0-d59d-4da7-b52d-f4e7b87cc7ca admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:06.802659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:06.803260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:06.803538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:06.803811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:06.842859 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:06.861349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:06.862827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:06.871618 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4103/8276] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:06.992626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:07.005198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d73e1-604a-48b5-b32b-672176b4bdb4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:07.008843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:07.009437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:07.009742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:07.010012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:07.019126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:07.020592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbabcbe-d578-43dc-9a69-ff90281a2b82 service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:07.029498 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4174/8277] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:07.358820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:07.423778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017d319-8b43-4b9e-abf7-9fe46ee038b2 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:07.428346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:07.428969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:07.429248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:07.429519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] PATH_INFO: `/v3/users/ed1eec0df5da4e5281e630005d1ca682` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:07.430670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed1eec0df5da4e5281e630005d1ca682)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:07.432632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:07.479578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed1eec0df5da4e5281e630005d1ca682'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:07.480227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed1eec0df5da4e5281e630005d1ca682'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:07.488201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed1eec0df5da4e5281e630005d1ca682'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:07.495440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed1eec0df5da4e5281e630005d1ca682'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:07.496087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed1eec0df5da4e5281e630005d1ca682'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:07.512873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4104/8278] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:07 2022] DELETE /identity/v3/users/ed1eec0df5da4e5281e630005d1ca682 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:07.517876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:07.518162 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:07.518439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:07.538995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:07.540196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:07.598425 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4175/8279] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:07.608348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:07.610448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:07.672988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a97a23-1391-4d7e-823d-8adaa5bf4840 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:07.676844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:07.677439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:07.677713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:07.677980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:07.687389 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:07.689011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:07.698575 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4105/8280] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:07.968327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:07.970202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:07.979053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:07.983227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb007-d20a-4402-8c96-9b67daa457b9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:07.991381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:07.992322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:07.992738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:07.993559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] PATH_INFO: `/v3/projects/d3d8c2e4e34e4105bf5747711fcedf6b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:07.994629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d8c2e4e34e4105bf5747711fcedf6b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:07.996913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.044188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d8c2e4e34e4105bf5747711fcedf6b', 'request_id': 'req-8ce20a2a-1480-495f-8908-903f439d5231'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.050304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d8c2e4e34e4105bf5747711fcedf6b', 'request_id': 'req-8ce20a2a-1480-495f-8908-903f439d5231'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.056632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d8c2e4e34e4105bf5747711fcedf6b', 'request_id': 'req-8ce20a2a-1480-495f-8908-903f439d5231'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.069800 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] The token cache is being invalidate because project d3d8c2e4e34e4105bf5747711fcedf6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:08.070088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.070821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.072379 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4176/8281] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:07 2022] DELETE /identity/v3/projects/d3d8c2e4e34e4105bf5747711fcedf6b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:08.076324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.081259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:08.088547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5ef7-f737-42d8-9e54-b8a60c6253f5 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.147068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.147695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.147977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.148255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] PATH_INFO: `/v3/users/99c58044cc564946b9cf8d65ddb89f46` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.149373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c58044cc564946b9cf8d65ddb89f46)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.151377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.197328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c58044cc564946b9cf8d65ddb89f46'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.198004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c58044cc564946b9cf8d65ddb89f46'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.205503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c58044cc564946b9cf8d65ddb89f46'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.212618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c58044cc564946b9cf8d65ddb89f46'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.213174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8ea79bdf-4ef9-423b-9087-b5bd2312ee5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c58044cc564946b9cf8d65ddb89f46'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.230670 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4106/8282] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:08 2022] DELETE /identity/v3/users/99c58044cc564946b9cf8d65ddb89f46 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:08.506382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.507995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:08.523231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce20a2a-1480-495f-8908-903f439d5231 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.580865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.581502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.581786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.582061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] PATH_INFO: `/v3/projects/e14176f90d294f8da9d1bfb0921afe01` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.583214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14176f90d294f8da9d1bfb0921afe01)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.585158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.631593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14176f90d294f8da9d1bfb0921afe01', 'request_id': 'req-65198d91-2a0c-481d-bd57-58d63e2faa26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.637400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14176f90d294f8da9d1bfb0921afe01', 'request_id': 'req-65198d91-2a0c-481d-bd57-58d63e2faa26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.644174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14176f90d294f8da9d1bfb0921afe01', 'request_id': 'req-65198d91-2a0c-481d-bd57-58d63e2faa26'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.657071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] The token cache is being invalidate because project e14176f90d294f8da9d1bfb0921afe01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:08.657379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.658006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-65198d91-2a0c-481d-bd57-58d63e2faa26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:08.659548 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4177/8283] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:08 2022] DELETE /identity/v3/projects/e14176f90d294f8da9d1bfb0921afe01 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:08.665362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.665838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.665967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.687981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:08.689684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:08.748246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4107/8284] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:08.757277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.757504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.757776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.777740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:08.778977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:08.787691 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4178/8285] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:08.792148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.802990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef63e2-6654-44a6-96b6-a874d3093065 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.807649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.808279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.808562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.808840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.809513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:08.809820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.811482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.823897 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4108/8286] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:08.828359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.837911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61528fd0-00d5-4e1b-9739-d14de9e0c612 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.842633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.843221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.843624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.843978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.844606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.846576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.855967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:08.876052 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4179/8287] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:08 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:08.880340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.890763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33852be9-43d1-4fa5-8082-f4538cf54c54 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.896116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:08.948948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.949561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.949846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.950121 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.950788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.952726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:08.973884 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4109/8288] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:08 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:08.978008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:08.989756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42cb00-c49c-4508-b83d-3da8b6e5681f admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:08.994468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:08.995090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:08.995363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:08.995634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:08.996268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:08.997848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.008051 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4180/8289] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:09.011910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:09.022435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7040a-abac-41ac-9685-c285b45cf474 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:09.027742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:09.028341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.028623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.028906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.029544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:09.031938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cda0c-cbd5-4738-8c4e-e5a6983074f8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.042664 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4110/8290] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:09.047374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:09.057545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7662a02-3ce8-46fb-abae-b6a056775097 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:09.062291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:09.063088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.063365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.063639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] PATH_INFO: `/v3/projects/013b98bcbeb445a084766d7f150fa514/users/6d6a6ba8f0d943cfa6415bb6a9477b10/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.078207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013b98bcbeb445a084766d7f150fa514, user_id=6d6a6ba8f0d943cfa6415bb6a9477b10, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:09.080940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.094363 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4181/8291] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:09 2022] PUT /identity/v3/projects/013b98bcbeb445a084766d7f150fa514/users/6d6a6ba8f0d943cfa6415bb6a9477b10/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:09.106446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.106764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.107043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.136494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] MFA Rules not processed for user `6d6a6ba8f0d943cfa6415bb6a9477b10`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:09.137709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:09.202062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4111/8292] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:09.213677 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:09.215449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:09.275450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b784fe-cf40-4608-8ff4-9ae0c6e52485 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:09.279071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:09.279678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.279959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.280234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.289242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:09.290701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f281ec-f849-4731-94b8-12adf64c3d3e service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.299550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4182/8293] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:09.659675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:09.723481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205a58e-56ea-456f-aee6-1cd561d2b473 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:09.728198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:09.728818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.729117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.729409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] PATH_INFO: `/v3/users/6d6a6ba8f0d943cfa6415bb6a9477b10` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.730593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6a6ba8f0d943cfa6415bb6a9477b10)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:09.732531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.753290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:09.778965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6a6ba8f0d943cfa6415bb6a9477b10'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:09.779617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6a6ba8f0d943cfa6415bb6a9477b10'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:09.787408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6a6ba8f0d943cfa6415bb6a9477b10'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:09.794589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6a6ba8f0d943cfa6415bb6a9477b10'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:09.795224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6a6ba8f0d943cfa6415bb6a9477b10'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:09.812892 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4112/8294] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:09 2022] DELETE /identity/v3/users/6d6a6ba8f0d943cfa6415bb6a9477b10 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:09.817436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.817724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.817998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.837411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:09.838665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:09.899504 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4183/8295] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:09.909288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:09.910738 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:09.969981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15501969-1c1a-4a16-ad00-6d2af8ca26fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:09.973714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:09.974308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:09.974625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:09.974901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:09.983841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:09.986046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b283c4fe-6abc-43bb-bcb6-6ff418155c50 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:09.995559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4113/8296] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:10.261066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.262523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:10.276626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8aee7-82e3-40a5-ba3d-a94fdbe6e2c1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.281449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.282081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.282363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.282673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] PATH_INFO: `/v3/projects/013b98bcbeb445a084766d7f150fa514` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.283805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013b98bcbeb445a084766d7f150fa514)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.285712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.333184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013b98bcbeb445a084766d7f150fa514', 'request_id': 'req-3d648065-5357-4230-8c60-0b832b85e736'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:10.335145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:10.339762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013b98bcbeb445a084766d7f150fa514', 'request_id': 'req-3d648065-5357-4230-8c60-0b832b85e736'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:10.346829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013b98bcbeb445a084766d7f150fa514', 'request_id': 'req-3d648065-5357-4230-8c60-0b832b85e736'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:10.358647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] The token cache is being invalidate because project 013b98bcbeb445a084766d7f150fa514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:10.358930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:10.360367 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3d648065-5357-4230-8c60-0b832b85e736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:10.361905 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4184/8297] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:10 2022] DELETE /identity/v3/projects/013b98bcbeb445a084766d7f150fa514 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:10.367057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.367347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.367625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.388093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:10.389437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:10.449900 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4114/8298] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:10.458003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.458291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.458606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.479761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:10.480972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:10.489775 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4185/8299] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:10.494066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.504740 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939634-5ebe-479d-a0bf-62449ab426bc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.509291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.509874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.510148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.510444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.511123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:10.511421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.513027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.525400 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4115/8300] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:10.529647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.539387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583f412-7bb3-40aa-b855-a51a7f11781c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.544053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.544637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.544911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.545178 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.545832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.548074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.575749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4186/8301] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:10.580741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.590801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69ebbc-7c3e-44ca-80ef-48ee87759c41 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.614598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:10.649218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.649847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.650128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.650400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.651080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.653034 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.674127 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4116/8302] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:10.678272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.689301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf311-8a9e-431b-9e2f-9c131cdb24ba admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.694387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.695010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.695290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.695561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.696199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.697779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.709032 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4187/8303] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:10.712884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.722995 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd00e7-f1ee-4c86-aea6-f9aa19cdc861 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.727634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.728202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.728485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.728756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.729382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.731072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85335db-e800-4ace-b6bd-27996c304787 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.742062 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4117/8304] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:10.746238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.757263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817f630-02b1-489e-9f75-9429e587a5e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.761961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.762645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.762930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.763207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] PATH_INFO: `/v3/projects/72a23c63ba114a2892ec6c29ec539284/users/3d6fd9f1e4544dda9edd20e487932fce/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.778385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a23c63ba114a2892ec6c29ec539284, user_id=3d6fd9f1e4544dda9edd20e487932fce, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.781092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:10.794030 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4188/8305] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:10 2022] PUT /identity/v3/projects/72a23c63ba114a2892ec6c29ec539284/users/3d6fd9f1e4544dda9edd20e487932fce/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:10.805543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.805828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.806106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.835110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] MFA Rules not processed for user `3d6fd9f1e4544dda9edd20e487932fce`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:10.836346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:10.900676 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4118/8306] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:28:10.905145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:10.967911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e8d12-ab01-421f-afad-ac2ba88ab789 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:10.972781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:10.973380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:10.973664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:10.973936 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:10.974595 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:10.976506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.004198 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4189/8307] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:10 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:11.008292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.019640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f80ca3-6a5b-4004-b97b-f22e8aea7cc8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.076782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.077367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.077644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.077978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.078687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.080605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.101399 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4119/8308] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:11 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:11.105429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.116614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b3113-ec23-4d55-9726-68cf22020445 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.121341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.121940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.122221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.122516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.123169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.124807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.135945 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4190/8309] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:11.140055 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.150423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f05b0-2d3c-4996-a964-50669f1bcc88 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.155236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.155816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.156697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.156982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] PATH_INFO: `/v3/projects/c3724a8b40cb430a98b4047c5d38005b/users/83593253c43e46c784202fa87db52d3d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.174749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3724a8b40cb430a98b4047c5d38005b, user_id=83593253c43e46c784202fa87db52d3d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.177405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.186318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:11.191841 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4120/8310] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:11 2022] PUT /identity/v3/projects/c3724a8b40cb430a98b4047c5d38005b/users/83593253c43e46c784202fa87db52d3d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:11.196011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.206003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86295c4-282a-46b0-af8e-d3abdf43ff94 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.252175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:11.263109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.263718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.264175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.264457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.265242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.267196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5301cb4-bc60-48ad-8ddd-356918a2cb64 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.277037 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4191/8311] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:11.281736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.292041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f727-611a-44ec-9c83-81326a156c12 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.297372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.298198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.298478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.298779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] PATH_INFO: `/v3/projects/c3724a8b40cb430a98b4047c5d38005b/users/83593253c43e46c784202fa87db52d3d/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.300706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3724a8b40cb430a98b4047c5d38005b, user_id=83593253c43e46c784202fa87db52d3d, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.303417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.313633 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4121/8312] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:11 2022] PUT /identity/v3/projects/c3724a8b40cb430a98b4047c5d38005b/users/83593253c43e46c784202fa87db52d3d/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:11.324669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.324956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.325231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.353191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] MFA Rules not processed for user `83593253c43e46c784202fa87db52d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:11.354384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:11.423337 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4192/8313] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:11.433142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:11.434879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:11.494662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3552f3-ef1e-4138-98b6-1b3bdc3063cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:11.498212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:11.498905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:11.499181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:11.499447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:11.555557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:11.557002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:11.566195 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4122/8314] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:12.354521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:12.356297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:12.449168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a49b5-0b1b-4749-92f6-388b575aa4d9 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:12.454354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:12.455008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:12.455286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:12.455559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:12.457414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:12.472447 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:12.474063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:12.489157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4193/8315] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:13.793836 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:13.795356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:13.810200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c3dd1-7fe2-4ec9-8921-a44273d2e2b5 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:13.816216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:13.816840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:13.817129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:13.817387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:13.827713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:13.829227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:13.839726 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4123/8316] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:15.111627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:15.125551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e267726-a58f-44c8-b613-f94bbc3a8f97 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:15.130831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:15.131425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:15.131705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:15.131975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:15.133666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:15.148112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:15.149707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:15.166162 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4194/8317] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:16.285843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:16.300349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821e224-2574-4b13-b41f-c09f7b03a40d service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:16.306153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:16.306792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:16.307070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:16.307341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:16.309100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:16.325300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:16.326932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb92717-2c7a-4505-b512-9cb3f671f323 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:16.341303 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4124/8318] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:16.356985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:16.410708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:16.438559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cfae3-19b8-4bdf-84ff-fe0484d80876 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:16.443448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:16.444075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:16.444355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:16.444632 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] PATH_INFO: `/v3/users/3d6fd9f1e4544dda9edd20e487932fce` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:16.445755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6fd9f1e4544dda9edd20e487932fce)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:16.447724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:16.498117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6fd9f1e4544dda9edd20e487932fce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.498833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6fd9f1e4544dda9edd20e487932fce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.506719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6fd9f1e4544dda9edd20e487932fce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.519136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6fd9f1e4544dda9edd20e487932fce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.519742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6fd9f1e4544dda9edd20e487932fce'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.532224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4195/8319] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:16 2022] DELETE /identity/v3/users/3d6fd9f1e4544dda9edd20e487932fce => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:16.536888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:16.537171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:16.537448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:16.556789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:16.558012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:16.617042 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4125/8320] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:16.626896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:16.628319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:16.691552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85df3e8-0236-4ed9-81fc-7ebbe9d73410 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:16.695528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:16.696158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:16.696438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:16.696711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:16.706836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:16.708273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:16.717966 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4196/8321] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:16.896316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:16.897810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:16.910794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da14ab2-5468-402b-bbbb-0010b12eef63 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:16.915463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:16.916097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:16.916376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:16.916653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] PATH_INFO: `/v3/projects/72a23c63ba114a2892ec6c29ec539284` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:16.917809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a23c63ba114a2892ec6c29ec539284)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:16.919764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:16.930787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:16.967050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a23c63ba114a2892ec6c29ec539284', 'request_id': 'req-29c68043-fa53-4538-b1c3-8f4e107346a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.973348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a23c63ba114a2892ec6c29ec539284', 'request_id': 'req-29c68043-fa53-4538-b1c3-8f4e107346a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.980727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a23c63ba114a2892ec6c29ec539284', 'request_id': 'req-29c68043-fa53-4538-b1c3-8f4e107346a2'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.992414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] The token cache is being invalidate because project 72a23c63ba114a2892ec6c29ec539284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:16.992697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.993342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:16.994886 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4126/8322] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:16 2022] DELETE /identity/v3/projects/72a23c63ba114a2892ec6c29ec539284 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:16.998780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.008882 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ace58-f194-4da1-9d06-9b61871fa275 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.067199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.067739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.068012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.068279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] PATH_INFO: `/v3/users/83593253c43e46c784202fa87db52d3d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.069378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83593253c43e46c784202fa87db52d3d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.071327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.073532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:17.122277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83593253c43e46c784202fa87db52d3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.122946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83593253c43e46c784202fa87db52d3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.130574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83593253c43e46c784202fa87db52d3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.143469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83593253c43e46c784202fa87db52d3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.144092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a868314f-e3ee-4971-ae9b-7b96706155d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83593253c43e46c784202fa87db52d3d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.155902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4197/8323] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:16 2022] DELETE /identity/v3/users/83593253c43e46c784202fa87db52d3d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:17.427686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.429147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:17.443003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c68043-fa53-4538-b1c3-8f4e107346a2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.499132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.499761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.500659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.501095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] PATH_INFO: `/v3/projects/c3724a8b40cb430a98b4047c5d38005b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.502218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3724a8b40cb430a98b4047c5d38005b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.504174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.549457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3724a8b40cb430a98b4047c5d38005b', 'request_id': 'req-1e46fd2d-53db-4780-8c4b-96e1915c3003'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.554863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3724a8b40cb430a98b4047c5d38005b', 'request_id': 'req-1e46fd2d-53db-4780-8c4b-96e1915c3003'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.562161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3724a8b40cb430a98b4047c5d38005b', 'request_id': 'req-1e46fd2d-53db-4780-8c4b-96e1915c3003'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.574414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] The token cache is being invalidate because project c3724a8b40cb430a98b4047c5d38005b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:17.574733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.575367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1e46fd2d-53db-4780-8c4b-96e1915c3003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:17.576820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4127/8324] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:17 2022] DELETE /identity/v3/projects/c3724a8b40cb430a98b4047c5d38005b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:17.581822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.582290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.582395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.601772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:17.603061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:17.663351 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4198/8325] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:28:17.671737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e85115-7968-4751-abe3-176727e76010 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.672156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e85115-7968-4751-abe3-176727e76010 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.672633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e85115-7968-4751-abe3-176727e76010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.692492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-26e85115-7968-4751-abe3-176727e76010 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:17.693678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-26e85115-7968-4751-abe3-176727e76010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:17.701470 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4128/8326] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:17.706177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.716717 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b8673-707d-4ab7-a2af-d352800cb320 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.722184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.722823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.723105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.723407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.724075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:17.724380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.725976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.737663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4199/8327] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:17.742542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e85115-7968-4751-abe3-176727e76010 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.751942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e85115-7968-4751-abe3-176727e76010 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.756689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.757279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.757559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.757834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.758520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.760453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.783178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:17.794536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4129/8328] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:17 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:28:17.795658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.805815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaab06-1feb-4d2f-9ed4-7eb93c8023b9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.864649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.865167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.865443 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.865762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.866379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.868381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.884799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:17.893893 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4200/8329] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:17 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:17.898300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.909659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f882c0-c8de-4701-a8ad-350da31b929a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.914303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.914929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.915210 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.915481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.916105 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.917710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.927828 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4130/8330] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:17.931875 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.942785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1635fd-9574-4ea6-9988-2b8630dcd5ad admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.947415 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.948002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.948277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.948543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.949167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:17.950778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826218c-4370-4a0b-b0c6-77a2a131fc4d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:17.962056 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4201/8331] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:17.966246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:17.976388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad48f3a-49db-4021-9d90-0593021e49bf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:17.981161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:17.981832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:17.982115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:17.982386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] PATH_INFO: `/v3/projects/025666d0fccd47c9bbbd611a39b12665/users/4b8aa079c7484be58bed920a0b12c950/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:17.997869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] RBAC: Authorizing `identity:create_grant(project_id=025666d0fccd47c9bbbd611a39b12665, user_id=4b8aa079c7484be58bed920a0b12c950, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.001113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.013957 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4131/8332] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:17 2022] PUT /identity/v3/projects/025666d0fccd47c9bbbd611a39b12665/users/4b8aa079c7484be58bed920a0b12c950/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:18.025030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213215a3-f53d-4137-a036-313804198715 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.025318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213215a3-f53d-4137-a036-313804198715 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.025592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213215a3-f53d-4137-a036-313804198715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.053031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-213215a3-f53d-4137-a036-313804198715 None None] MFA Rules not processed for user `4b8aa079c7484be58bed920a0b12c950`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:18.054238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-213215a3-f53d-4137-a036-313804198715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:18.118630 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4202/8333] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:18.123167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.184101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6558ae-3c5d-4313-88be-baee8e7c919e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.189515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.190116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.190397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.190708 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.191325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.193232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.218873 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4132/8334] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:18 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:18.223074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213215a3-f53d-4137-a036-313804198715 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.233916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213215a3-f53d-4137-a036-313804198715 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.289899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.290518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.290808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.291076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.291698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.293604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.314942 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4203/8335] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:18 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:18.318908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.329399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efdc45-e3e2-48c0-9268-ac92bf67dcbe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.333949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.334560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.334837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.335110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.335760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.337334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.347958 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4133/8336] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:18.351899 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.362823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56166338-24d6-4ef1-b59b-4c02f2e91c99 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.367630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.368273 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.368556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.368883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] PATH_INFO: `/v3/projects/4fd25cd61ff34455888f8ae243ec86a2/users/59b72eb04f904e088e52af58146e7620/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.384177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd25cd61ff34455888f8ae243ec86a2, user_id=59b72eb04f904e088e52af58146e7620, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.386876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.400233 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4204/8337] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:18 2022] PUT /identity/v3/projects/4fd25cd61ff34455888f8ae243ec86a2/users/59b72eb04f904e088e52af58146e7620/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:18.404186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.414244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfa1da-a9d7-485d-8ba8-eb4ab5c1c5bd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.470667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.471299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.471525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.471794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.472421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.474033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756fbf80-ec9c-45d7-90cf-d577c1edb3e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.483777 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4134/8338] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:18.488570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.499350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3df94-462a-4561-adb5-bf372ee071a8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.504675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.505323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.505604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.505880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] PATH_INFO: `/v3/projects/4fd25cd61ff34455888f8ae243ec86a2/users/59b72eb04f904e088e52af58146e7620/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.507815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd25cd61ff34455888f8ae243ec86a2, user_id=59b72eb04f904e088e52af58146e7620, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.510455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.520919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4205/8339] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:18 2022] PUT /identity/v3/projects/4fd25cd61ff34455888f8ae243ec86a2/users/59b72eb04f904e088e52af58146e7620/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:18.532150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.532424 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.532698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.560883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] MFA Rules not processed for user `59b72eb04f904e088e52af58146e7620`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:18.562080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:18.586123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:18.632597 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4135/8340] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:18.643611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:18.645417 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:18.679353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:18.708155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7df8ba-ee3c-4f35-894d-d1c9bbd03ec5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:18.711896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:18.712497 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:18.712795 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:18.713071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:18.722801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:18.724268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73233290-48eb-4109-b097-b39874beb0a6 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:18.733702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4206/8341] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:20.786115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:20.849615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265118-5c86-4d8e-9de6-cd5545253f6d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:20.859992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:20.860634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:20.860922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:20.861195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] PATH_INFO: `/v3/users/4b8aa079c7484be58bed920a0b12c950` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:20.862327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8aa079c7484be58bed920a0b12c950)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:20.864311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:20.912096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8aa079c7484be58bed920a0b12c950'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:20.912743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8aa079c7484be58bed920a0b12c950'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:20.920285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8aa079c7484be58bed920a0b12c950'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:20.927730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8aa079c7484be58bed920a0b12c950'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:20.928349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8aa079c7484be58bed920a0b12c950'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:20.945461 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4136/8342] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:20 2022] DELETE /identity/v3/users/4b8aa079c7484be58bed920a0b12c950 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:20.949948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:20.950519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:20.950519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:20.971316 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:20.972445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:21.032179 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4207/8343] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:21.042080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:21.043522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:21.087777 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:21.104715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c629b59-4dd8-47d7-929b-38cdf05b87a5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:21.108496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:21.109085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:21.109386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:21.109745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:21.119013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:21.120435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:21.130639 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4137/8344] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:21.403041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:21.404553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:21.416942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268465d-4a10-4847-933b-024f2e03a91a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:21.421901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:21.423346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:21.423819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:21.424092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] PATH_INFO: `/v3/projects/025666d0fccd47c9bbbd611a39b12665` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:21.425228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=025666d0fccd47c9bbbd611a39b12665)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:21.427942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:21.474603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '025666d0fccd47c9bbbd611a39b12665', 'request_id': 'req-75ee6f32-796e-47f0-b2cb-826eeb29dd99'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.479955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '025666d0fccd47c9bbbd611a39b12665', 'request_id': 'req-75ee6f32-796e-47f0-b2cb-826eeb29dd99'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.486918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '025666d0fccd47c9bbbd611a39b12665', 'request_id': 'req-75ee6f32-796e-47f0-b2cb-826eeb29dd99'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.499293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] The token cache is being invalidate because project 025666d0fccd47c9bbbd611a39b12665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:21.499580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.500271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.502485 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4208/8345] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:21 2022] DELETE /identity/v3/projects/025666d0fccd47c9bbbd611a39b12665 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:21.506394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:21.516018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8247433-6579-4b31-9689-3c3f5fcfd000 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:21.572880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:21.573503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:21.573889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:21.574169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] PATH_INFO: `/v3/users/59b72eb04f904e088e52af58146e7620` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:21.575404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b72eb04f904e088e52af58146e7620)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:21.577327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:21.622345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b72eb04f904e088e52af58146e7620'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.623028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b72eb04f904e088e52af58146e7620'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.634114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b72eb04f904e088e52af58146e7620'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.643051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b72eb04f904e088e52af58146e7620'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.644030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-f0776583-ba7d-4558-a03d-b5378abee07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b72eb04f904e088e52af58146e7620'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.661789 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4138/8346] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:21 2022] DELETE /identity/v3/users/59b72eb04f904e088e52af58146e7620 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:21.837807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:21.839366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:21.857628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee6f32-796e-47f0-b2cb-826eeb29dd99 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:21.911121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:21.919121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:21.919935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:21.920410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:21.920861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] PATH_INFO: `/v3/projects/4fd25cd61ff34455888f8ae243ec86a2` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:21.922013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd25cd61ff34455888f8ae243ec86a2)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:21.924561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:21.970454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd25cd61ff34455888f8ae243ec86a2', 'request_id': 'req-f4944093-28bb-4dde-b4fa-a0204152cf0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.975904 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd25cd61ff34455888f8ae243ec86a2', 'request_id': 'req-f4944093-28bb-4dde-b4fa-a0204152cf0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.981862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd25cd61ff34455888f8ae243ec86a2', 'request_id': 'req-f4944093-28bb-4dde-b4fa-a0204152cf0d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.995909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] The token cache is being invalidate because project 4fd25cd61ff34455888f8ae243ec86a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:21.996191 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.996812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f4944093-28bb-4dde-b4fa-a0204152cf0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:21.998284 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4209/8347] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:21 2022] DELETE /identity/v3/projects/4fd25cd61ff34455888f8ae243ec86a2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:22.003320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.003823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.003941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.024560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:22.025755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:22.085857 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4139/8348] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:22.093558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.093839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.094849 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.114693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:22.115889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:22.123749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4210/8349] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:22.128137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.133017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:22.139423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea79318-8f81-492d-9e6a-65ced413633b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.146028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.146659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.146937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.147208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.147841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:22.148136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.149775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.162003 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4140/8350] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:22.166863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.176670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab5869-1752-456b-8c9a-21439f74e41a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.181421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.182016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.182297 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.182606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.183226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.185144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.216584 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4211/8351] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:22 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:22.217748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.228497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a11b71-9380-4cb7-a246-88d1a3274613 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.283828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.284346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.284623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.284895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.285507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.287459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.307788 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4141/8352] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:22 2022] POST /identity/v3/users => generated 281 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:22.312033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.322517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c840c5f-a9dc-4b76-a6a1-493a44db0678 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.327170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.327760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.328036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.328306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.329756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.331410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.341135 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4212/8353] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:22.345916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.356029 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3471-7098-4ec6-ba2f-03ab6ff50e57 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.361239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.361825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.362100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.362367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.363078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.364634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bd301-087d-4d08-ad9f-5fc8478b64a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.375112 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4142/8354] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:22.379983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.390030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3865ac4-aa3d-41db-ad3e-510ca2116e8c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.395363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.396009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.396288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.396556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] PATH_INFO: `/v3/projects/d396661aaf5e421cadbe15263c585c9a/users/644b0a6987c84740b04cf061b501f596/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.412757 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d396661aaf5e421cadbe15263c585c9a, user_id=644b0a6987c84740b04cf061b501f596, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.415468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.429217 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4213/8355] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:22 2022] PUT /identity/v3/projects/d396661aaf5e421cadbe15263c585c9a/users/644b0a6987c84740b04cf061b501f596/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:22.440683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.440965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.441236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.468518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] MFA Rules not processed for user `644b0a6987c84740b04cf061b501f596`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:22.470246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:22.533387 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4143/8356] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:22.543318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:22.544995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:22.604477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae41e8-0662-45d7-9044-8dbded67a4ce admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:22.608105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:22.608698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:22.608971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:22.609242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:22.618864 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:22.620355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319d85b-6fc3-4410-95a2-c34b84ea74bb service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:22.629341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4214/8357] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:32.529857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:32.625169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:48.722218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:48.786765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceab071-69c9-4194-ba25-818069b9eff4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:48.791326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:48.791950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:48.792232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:48.792510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] PATH_INFO: `/v3/users/644b0a6987c84740b04cf061b501f596` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:48.793629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=644b0a6987c84740b04cf061b501f596)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:48.795592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:48.841864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644b0a6987c84740b04cf061b501f596'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:48.842671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644b0a6987c84740b04cf061b501f596'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:48.850550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644b0a6987c84740b04cf061b501f596'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:48.857585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644b0a6987c84740b04cf061b501f596'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:48.858212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644b0a6987c84740b04cf061b501f596'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:48.876072 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4144/8358] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:28:48 2022] DELETE /identity/v3/users/644b0a6987c84740b04cf061b501f596 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:48.880746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:48.881239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:48.881429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:48.900503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:48.901710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:48.961230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4215/8359] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:48.971572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:48.972986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.043088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d564384-e70f-4c10-b374-b49beabce35b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.046822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.047422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.047701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.047974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.057281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.058756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26907a3-5ebb-4409-b6b9-623ac92abc79 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.068112 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4145/8360] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:49.241312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.242831 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.260148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e3c1-9e8a-48bf-ad21-7a07abb31143 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.264846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.265429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.265709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.266078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] PATH_INFO: `/v3/projects/d396661aaf5e421cadbe15263c585c9a` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.267651 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d396661aaf5e421cadbe15263c585c9a)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.269560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.317018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd396661aaf5e421cadbe15263c585c9a', 'request_id': 'req-d89b1d1e-3333-4f16-acd3-1507b52efbdb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:49.322612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd396661aaf5e421cadbe15263c585c9a', 'request_id': 'req-d89b1d1e-3333-4f16-acd3-1507b52efbdb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:49.328921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd396661aaf5e421cadbe15263c585c9a', 'request_id': 'req-d89b1d1e-3333-4f16-acd3-1507b52efbdb'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:49.341822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] The token cache is being invalidate because project d396661aaf5e421cadbe15263c585c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:28:49.342138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:49.342785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-d89b1d1e-3333-4f16-acd3-1507b52efbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:28:49.344272 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4216/8361] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:28:49 2022] DELETE /identity/v3/projects/d396661aaf5e421cadbe15263c585c9a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:28:49.350212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.350520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.350830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.370434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:49.371710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.389171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:49.434416 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4146/8362] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:49.442287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.442603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.442884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.464299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:49.465516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.473523 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4217/8363] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:49.477680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.487770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c938e-1cd3-452d-bc11-1e1bb55f9a81 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.492349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.492936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.493214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.493485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.494166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:28:49.494446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.496069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.508702 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4147/8364] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:28:49.512930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.523254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895114b-6db4-4203-a09e-9e2266dd1dc5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.527851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.528437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.528714 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.528983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.529602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.531529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.557477 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4218/8365] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:28:49 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:49.561625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.572369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae253-9350-45f9-8ee8-12efc2f43163 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.627784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.628380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.628687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.628970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.629609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.631589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.653026 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4148/8366] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:28:49 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:28:49.657197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.667536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9daecd-900e-4da2-ba83-c9c76b84ac01 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.672483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.673072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.673352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.673836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.674474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.676088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.686458 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4219/8367] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:49.691058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.701338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ff5d3-66f1-4f42-bba3-cce25d2b9aa4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.706728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.707320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.707594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.707864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.708486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.710060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5d735-1283-45ee-a183-9e80a7af9b1f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.720596 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4149/8368] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:28:49.724727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.734699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee616f-768b-4b70-beca-70b122dd9b71 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.740023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.740605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.740883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.741156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] PATH_INFO: `/v3/projects/85a69b3fe4ec40baa28c2a6606396f0b/users/1603acd4363e41d19095097965885183/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.756101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a69b3fe4ec40baa28c2a6606396f0b, user_id=1603acd4363e41d19095097965885183, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.758781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.771585 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4220/8369] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:28:49 2022] PUT /identity/v3/projects/85a69b3fe4ec40baa28c2a6606396f0b/users/1603acd4363e41d19095097965885183/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:28:49.782061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.782572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.782699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.811556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] MFA Rules not processed for user `1603acd4363e41d19095097965885183`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:28:49.812741 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.876212 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4150/8370] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:28:49.893055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:28:49.895066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:28:49.918543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:49.959445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbeb8d-e29b-4082-a277-93988e73165c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:28:49.963206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:28:49.963888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:28:49.964165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:28:49.964436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:28:49.974730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:28:49.976168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e57d54-f7c1-428b-b212-0dd4e5312b00 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:28:49.985144 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4221/8371] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:28:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:28:59.873392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:28:59.980975 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=17 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:32.028502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.090894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193317b9-dd99-4091-b3f2-e640d2da89e7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.095543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.096152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.096403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.096675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] PATH_INFO: `/v3/users/1603acd4363e41d19095097965885183` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.097926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1603acd4363e41d19095097965885183)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.100460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.149330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1603acd4363e41d19095097965885183'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.150001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1603acd4363e41d19095097965885183'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.157223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1603acd4363e41d19095097965885183'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.165063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1603acd4363e41d19095097965885183'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.165723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1603acd4363e41d19095097965885183'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.184297 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4151/8372] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:32 2022] DELETE /identity/v3/users/1603acd4363e41d19095097965885183 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:32.189030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871287-311b-43f5-b62e-83d028d31385 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.189483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871287-311b-43f5-b62e-83d028d31385 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.189593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871287-311b-43f5-b62e-83d028d31385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.208652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-63871287-311b-43f5-b62e-83d028d31385 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:32.209845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-63871287-311b-43f5-b62e-83d028d31385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:32.268366 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4222/8373] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:32.278669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.280081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:32.340414 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f51e8-f9f3-4e87-9d08-a210690a5398 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.344099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.344690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.344967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.345242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.355056 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.356493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5291c-816a-419a-a578-46b5c41e3287 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.366260 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4152/8374] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:32.543916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871287-311b-43f5-b62e-83d028d31385 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.546158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-63871287-311b-43f5-b62e-83d028d31385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:32.563609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871287-311b-43f5-b62e-83d028d31385 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.569585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.570306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.570634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.570884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] PATH_INFO: `/v3/projects/85a69b3fe4ec40baa28c2a6606396f0b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.572355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a69b3fe4ec40baa28c2a6606396f0b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.575000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.621653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a69b3fe4ec40baa28c2a6606396f0b', 'request_id': 'req-703900a8-4910-4f3b-b73a-fc8033d64ebc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.626963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a69b3fe4ec40baa28c2a6606396f0b', 'request_id': 'req-703900a8-4910-4f3b-b73a-fc8033d64ebc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.634256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a69b3fe4ec40baa28c2a6606396f0b', 'request_id': 'req-703900a8-4910-4f3b-b73a-fc8033d64ebc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.652005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] The token cache is being invalidate because project 85a69b3fe4ec40baa28c2a6606396f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:32.652300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.652976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-703900a8-4910-4f3b-b73a-fc8033d64ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:32.654517 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4223/8375] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:32 2022] DELETE /identity/v3/projects/85a69b3fe4ec40baa28c2a6606396f0b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:32.659904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.660190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.660491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.682478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:32.683780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:32.701711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:32.745991 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4153/8376] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:32.753529 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a759-0821-484c-b302-ba14375bdfab None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.753815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a759-0821-484c-b302-ba14375bdfab None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.754090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a759-0821-484c-b302-ba14375bdfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.775689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3748a759-0821-484c-b302-ba14375bdfab None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:32.776898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3748a759-0821-484c-b302-ba14375bdfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:32.784827 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4224/8377] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:32.789249 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.799116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee2570-0e53-40e8-b616-fb517e5c3f07 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.804138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.804727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.805004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.805273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.805902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:32.806199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.807827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.819690 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4154/8378] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:32.823897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748a759-0821-484c-b302-ba14375bdfab None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.833887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748a759-0821-484c-b302-ba14375bdfab None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.838830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.839434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.839713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.839987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.840609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.842583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.869633 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4225/8379] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:32 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:32.873739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.884239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b37e87-598e-4102-a1a3-57f222f42ecb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.940007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.940606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.940889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.941165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.941799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.944279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.965581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4155/8380] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:32 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:32.969822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:32.980435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f42a95-1374-4110-a25d-8a167f8c5295 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:32.985103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:32.985700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:32.985981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:32.986253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:32.986930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:32.988518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:32.998475 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4226/8381] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:33.002376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:33.013106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94c21-8b02-4604-8886-e744d5c041c7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:33.017992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:33.018620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:33.018901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:33.019176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:33.019804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:33.022038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed476ce9-9ebe-4b36-982b-084d7d1269f5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:33.032796 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4156/8382] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:33.037007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:33.047054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00abec-789f-4e8a-8c04-c496e6f39cc6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:33.052357 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:33.053002 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:33.053280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:33.053545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] PATH_INFO: `/v3/projects/d19a64cd5de04b92b05b6772ac3602cb/users/f1693dcc0eee47c587437b51cd5c5eb5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:33.068467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19a64cd5de04b92b05b6772ac3602cb, user_id=f1693dcc0eee47c587437b51cd5c5eb5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:33.071175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:33.084836 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4227/8383] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:33 2022] PUT /identity/v3/projects/d19a64cd5de04b92b05b6772ac3602cb/users/f1693dcc0eee47c587437b51cd5c5eb5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:33.095799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:33.096115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:33.096384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:33.125710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] MFA Rules not processed for user `f1693dcc0eee47c587437b51cd5c5eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:33.126860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:33.190342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4157/8384] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:33.201814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:33.203676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:33.225645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:33.265068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176d4dd-1ec4-4d12-9ecd-c489ef49261b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:33.268770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:33.269377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:33.269654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:33.269963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:33.279637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:33.281062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ec9e7-3ade-4d1f-b2d8-aa808a0d43b7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:33.290247 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4228/8385] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:39.730077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:39.749149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:39.793582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22df2-7f03-4e94-ae1c-13ded4f3535f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:39.798070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:39.798723 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:39.799006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:39.799276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] PATH_INFO: `/v3/users/f1693dcc0eee47c587437b51cd5c5eb5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:39.800412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1693dcc0eee47c587437b51cd5c5eb5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:39.803010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:39.848650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1693dcc0eee47c587437b51cd5c5eb5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:39.849431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1693dcc0eee47c587437b51cd5c5eb5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:39.856319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1693dcc0eee47c587437b51cd5c5eb5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:39.863099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1693dcc0eee47c587437b51cd5c5eb5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:39.863730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1693dcc0eee47c587437b51cd5c5eb5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:39.881554 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4158/8386] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:39 2022] DELETE /identity/v3/users/f1693dcc0eee47c587437b51cd5c5eb5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:39.886161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:39.886659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:39.886751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:39.907015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:39.908260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:39.969113 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4229/8387] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:39.979409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:39.980801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.041632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045e2da-b242-40a5-98d4-fe4e3a79755e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.045418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.046016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.046294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.046617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.056523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.057955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d59344-7c99-487c-b727-2143d119e825 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.068033 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4159/8388] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:40.240916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.242387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.255514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49bb0a-b3ca-4926-a5af-38147d3787d5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.260379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.261049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.261327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.261614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] PATH_INFO: `/v3/projects/d19a64cd5de04b92b05b6772ac3602cb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.262785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19a64cd5de04b92b05b6772ac3602cb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.264753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.313042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19a64cd5de04b92b05b6772ac3602cb', 'request_id': 'req-4c22dc20-e385-4679-8934-f6f257c6f98e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:40.318555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19a64cd5de04b92b05b6772ac3602cb', 'request_id': 'req-4c22dc20-e385-4679-8934-f6f257c6f98e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:40.324747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19a64cd5de04b92b05b6772ac3602cb', 'request_id': 'req-4c22dc20-e385-4679-8934-f6f257c6f98e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:40.337029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] The token cache is being invalidate because project d19a64cd5de04b92b05b6772ac3602cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:40.337307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:40.337951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-4c22dc20-e385-4679-8934-f6f257c6f98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:40.339488 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4230/8389] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:40 2022] DELETE /identity/v3/projects/d19a64cd5de04b92b05b6772ac3602cb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:40.344377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.346014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.346297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.368490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:40.369714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.412383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:40.435733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4160/8390] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:40.443696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.443978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.444246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.463641 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:40.464827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.472655 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4231/8391] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:29:40.477569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.487458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d65ba0-8497-4192-bdc6-6c06ff1463d2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.492982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.493576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.493855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.494126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.494808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:40.495110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.496705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.508131 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4161/8392] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:40.512501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.521843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa458fa7-fb51-4c5e-962e-3c26750dead6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.526891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.527478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.527758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.528067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.528691 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.531125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.578815 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4232/8393] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:40 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:29:40.581699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.592378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41528156-9283-41ec-98a7-fc734dfae07f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.649819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.650420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.650729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.651003 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.651632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.653560 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.675581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4162/8394] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:40 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:40.680154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.690472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b1164-faac-4b7a-9f2e-09a47fc3b40e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.695577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.696159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.696433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.696702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.697327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.698949 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.709450 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4233/8395] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:40.713918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.724065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f425-f02c-4e0a-9588-d424502bf3cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.728841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.729431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.729711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.729987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.730660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.732298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc802b7-4d6f-46d5-96c6-55fe2bf3c238 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.743547 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4163/8396] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:40.747577 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.757982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029c5c7-4a84-43ef-b777-2909f8bd6ec4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.762676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.763310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.763585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.763853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] PATH_INFO: `/v3/projects/b143f5bc404e481c983e2349e6da10da/users/b01a36262ed84a6fa28d8df3e4372106/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.766035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:40.781393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b143f5bc404e481c983e2349e6da10da, user_id=b01a36262ed84a6fa28d8df3e4372106, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.784100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.797136 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4234/8397] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:40 2022] PUT /identity/v3/projects/b143f5bc404e481c983e2349e6da10da/users/b01a36262ed84a6fa28d8df3e4372106/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:40.808211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.808499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.808804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.837710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] MFA Rules not processed for user `b01a36262ed84a6fa28d8df3e4372106`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:40.838947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.903043 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4164/8398] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:40.914409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:40.916240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:40.975340 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024d96-fde4-4608-9274-c60e8d2141ca admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:40.978976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:40.979567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:40.979842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:40.980111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:40.988390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:40.990177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f10db6-ba75-4bea-bbf2-1d17beb0d7e2 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:40.998912 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4235/8399] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:41.314823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:41.354346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:41.379696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5640368-b9c8-4854-8595-926fc5d34f6d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:41.384707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:41.385331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:41.385612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:41.385887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] PATH_INFO: `/v3/users/b01a36262ed84a6fa28d8df3e4372106` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:41.387146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01a36262ed84a6fa28d8df3e4372106)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:41.389074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:41.436051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01a36262ed84a6fa28d8df3e4372106'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:41.436699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01a36262ed84a6fa28d8df3e4372106'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:41.444453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01a36262ed84a6fa28d8df3e4372106'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:41.451344 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01a36262ed84a6fa28d8df3e4372106'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:41.451974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01a36262ed84a6fa28d8df3e4372106'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:41.468908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4165/8400] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:41 2022] DELETE /identity/v3/users/b01a36262ed84a6fa28d8df3e4372106 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:41.473296 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:41.473622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:41.473897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:41.493475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:41.494715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:41.553033 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4236/8401] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:41.562615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:41.564017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:41.623997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5830ec-c518-4118-9ead-ed796194438f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:41.627897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:41.628512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:41.628778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:41.629498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:41.639086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:41.640545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326271c-fd7f-4ac9-89bc-6170c3e62d4a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:41.651139 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4166/8402] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:41.924615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:41.926081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:41.940699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a2551-fde5-43c5-b449-ba5511c6b5c1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:41.947274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:41.947920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:41.948174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:41.948449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] PATH_INFO: `/v3/projects/b143f5bc404e481c983e2349e6da10da` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:41.949559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b143f5bc404e481c983e2349e6da10da)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:41.951603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:41.999925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b143f5bc404e481c983e2349e6da10da', 'request_id': 'req-284685f5-49db-492a-98b6-750141be5206'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:42.006135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b143f5bc404e481c983e2349e6da10da', 'request_id': 'req-284685f5-49db-492a-98b6-750141be5206'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:42.012907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b143f5bc404e481c983e2349e6da10da', 'request_id': 'req-284685f5-49db-492a-98b6-750141be5206'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:42.026044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] The token cache is being invalidate because project b143f5bc404e481c983e2349e6da10da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:42.026333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:42.027004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-284685f5-49db-492a-98b6-750141be5206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:42.028560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4237/8403] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:41 2022] DELETE /identity/v3/projects/b143f5bc404e481c983e2349e6da10da => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:42.033757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.034222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.034315 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.055956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:42.057188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:42.107518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:42.123291 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4167/8404] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:42.218455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.218707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.218990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.242662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:42.243878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:42.251823 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4238/8405] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:42.256788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.268151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62908-2f83-4832-921d-d3bc9ba5702a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.273746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.274070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.274351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.274920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.275575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:42.275875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.277498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.289560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4168/8406] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:42.293897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.303331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:42.311307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de105-8f6e-4874-9760-d6ec5f8ab122 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.318206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.319249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.319997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.320282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.320921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.323137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.351600 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4239/8407] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:42.356001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.366829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597031-f7cd-4383-b85e-9844e99e572b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.425339 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.425940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.426214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.426485 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.427734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.429648 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.450984 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4169/8408] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:42.455014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.465523 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f76e24-212f-44b6-95c8-aab0b92037f8 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.470256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.470820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.471099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.471371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.472001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.474248 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.484317 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4240/8409] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:42.488375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.500693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08661f21-84d7-42c8-b9c0-1970d827d57d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.507109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.507768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.508131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.508420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.509137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.511179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bb329-9538-431c-92d7-f82a8f060485 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.522770 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4170/8410] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:42.527644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.538315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21067b2-fb8e-48fd-9cd0-5c268361ce89 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.543034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.543678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.543956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.544228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] PATH_INFO: `/v3/projects/c116eb55424a4ebca7725867f99bfb18/users/bc24da99f6174f679e157761f8534c1b/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.559642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c116eb55424a4ebca7725867f99bfb18, user_id=bc24da99f6174f679e157761f8534c1b, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.562271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.576210 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4241/8411] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:42 2022] PUT /identity/v3/projects/c116eb55424a4ebca7725867f99bfb18/users/bc24da99f6174f679e157761f8534c1b/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:42.587227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.587515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.587797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.614919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] MFA Rules not processed for user `bc24da99f6174f679e157761f8534c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:42.616452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:42.682651 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4171/8412] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:42.687103 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.749138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5bca1-b306-4b71-8c4d-6767f863b782 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.753698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.754282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.754585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.755413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.756226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.758116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.784879 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4242/8413] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:42.789313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.800116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2e94-c93d-41d7-8cc4-dfdc6941cba9 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.847792 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:42.857967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.858603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.858926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.859206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.859820 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.861754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.882880 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4172/8414] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:42 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:42.887054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.898134 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49285e69-80ef-4cad-ba9b-1c490d3b1b4a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.903208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.903798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.904078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.904348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.904978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.906618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.918850 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4243/8415] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:42.923152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.933542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1beb6-f0c4-4375-844e-4c00ba7bf632 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:42.938225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:42.938917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:42.939201 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:42.939478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] PATH_INFO: `/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab/users/53932313bc1d42b3aa10b640dfe8462c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:42.953976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dd9f2047a44955996157e7a2b5e7ab, user_id=53932313bc1d42b3aa10b640dfe8462c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:42.956679 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:42.969849 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4173/8416] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:42 2022] PUT /identity/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab/users/53932313bc1d42b3aa10b640dfe8462c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:42.974014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:42.984216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e9660-cd25-4dd9-be20-db6b6264dfed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:43.041670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:43.042194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:43.042476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:43.042803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:43.043438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:43.045041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a3cc5-b10a-49dc-bebf-f54487b38ee8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:43.055346 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4244/8417] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:43.059629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:43.071041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379c2f5-0c23-4c50-8f94-fc90a7de0b8a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:43.075634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:43.076269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:43.076544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:43.076816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] PATH_INFO: `/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab/users/53932313bc1d42b3aa10b640dfe8462c/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:43.078757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dd9f2047a44955996157e7a2b5e7ab, user_id=53932313bc1d42b3aa10b640dfe8462c, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:43.081376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:43.091605 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4174/8418] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:43 2022] PUT /identity/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab/users/53932313bc1d42b3aa10b640dfe8462c/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:43.102091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:43.102379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:43.102696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:43.132046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] MFA Rules not processed for user `53932313bc1d42b3aa10b640dfe8462c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:43.133303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:43.139370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:43.218749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4245/8419] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:43.231697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:43.233496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:43.304542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c12aa-aa56-499f-a853-f79f39d8f26c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:43.308227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:43.308824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:43.309103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:43.309369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:43.318544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:43.319974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:43.329381 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4175/8420] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:43.419281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:43.432204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062956e-b19b-4a8b-8b8e-fa832ce8a6e3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:43.435917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:43.436517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:43.436798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:43.437070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:43.493669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:43.495145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa21fc-4abe-425b-ba73-7ca93c72db55 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:43.504193 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4246/8421] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:45.887816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:45.950571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb04f70-917f-4b66-9483-0167dfdc8fce service cinder] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:45.955126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:45.955745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:45.956027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:45.956304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] PATH_INFO: `/v3/users/bc24da99f6174f679e157761f8534c1b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:45.957413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc24da99f6174f679e157761f8534c1b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:45.959377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:46.000120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:46.006895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc24da99f6174f679e157761f8534c1b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.007942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc24da99f6174f679e157761f8534c1b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.015720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc24da99f6174f679e157761f8534c1b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.022914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc24da99f6174f679e157761f8534c1b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.023574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc24da99f6174f679e157761f8534c1b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.042107 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4176/8422] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:45 2022] DELETE /identity/v3/users/bc24da99f6174f679e157761f8534c1b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:46.046715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:46.047168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:46.047277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:46.066827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:46.068709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:46.126663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4247/8423] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:46.136869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:46.138354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:46.199525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e81b69-3d97-44cb-a3cd-ddc55ccbec07 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:46.203267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:46.203867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:46.204150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:46.204425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:46.213563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:46.215043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:46.225351 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4177/8424] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:46.498938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:46.500384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:46.513780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a1a8e-45a4-4903-83c6-b47d7ed69a8c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:46.518535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:46.519168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:46.519444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:46.519744 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] PATH_INFO: `/v3/projects/c116eb55424a4ebca7725867f99bfb18` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:46.522224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c116eb55424a4ebca7725867f99bfb18)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:46.524157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:46.563154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:46.574612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c116eb55424a4ebca7725867f99bfb18', 'request_id': 'req-3ee72de2-8354-4cb2-beda-8de8361d029d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.580311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c116eb55424a4ebca7725867f99bfb18', 'request_id': 'req-3ee72de2-8354-4cb2-beda-8de8361d029d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.587146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c116eb55424a4ebca7725867f99bfb18', 'request_id': 'req-3ee72de2-8354-4cb2-beda-8de8361d029d'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.600074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] The token cache is being invalidate because project c116eb55424a4ebca7725867f99bfb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:46.600368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.600995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.602554 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4248/8425] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:46 2022] DELETE /identity/v3/projects/c116eb55424a4ebca7725867f99bfb18 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:46.606450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:46.616336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d102692-6819-4fc5-b018-bd158862df72 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:46.674524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:46.675360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:46.675735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:46.676045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] PATH_INFO: `/v3/users/53932313bc1d42b3aa10b640dfe8462c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:46.677609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53932313bc1d42b3aa10b640dfe8462c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:46.679893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:46.726203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53932313bc1d42b3aa10b640dfe8462c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.726932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53932313bc1d42b3aa10b640dfe8462c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.734471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53932313bc1d42b3aa10b640dfe8462c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.743503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53932313bc1d42b3aa10b640dfe8462c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.744171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ebcd498b-6e63-4afb-b600-cdd83005ed03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53932313bc1d42b3aa10b640dfe8462c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:46.745290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:46.763887 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4178/8426] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:46 2022] DELETE /identity/v3/users/53932313bc1d42b3aa10b640dfe8462c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:29:47.042086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.043940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:47.059896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee72de2-8354-4cb2-beda-8de8361d029d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.119917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.120553 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.120835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.121111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] PATH_INFO: `/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.122239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dd9f2047a44955996157e7a2b5e7ab)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.124194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.171727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dd9f2047a44955996157e7a2b5e7ab', 'request_id': 'req-99e59481-bb46-4e1e-96c5-3dd1b7c59497'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:47.177749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dd9f2047a44955996157e7a2b5e7ab', 'request_id': 'req-99e59481-bb46-4e1e-96c5-3dd1b7c59497'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:47.183911 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dd9f2047a44955996157e7a2b5e7ab', 'request_id': 'req-99e59481-bb46-4e1e-96c5-3dd1b7c59497'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:47.196840 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] The token cache is being invalidate because project c8dd9f2047a44955996157e7a2b5e7ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:47.197129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:47.197823 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-99e59481-bb46-4e1e-96c5-3dd1b7c59497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:47.199394 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4249/8427] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:47 2022] DELETE /identity/v3/projects/c8dd9f2047a44955996157e7a2b5e7ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:47.204206 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.204653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.204754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.224772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:47.225916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:47.284086 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4179/8428] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:47.292048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.292331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.292608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.313223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:47.314413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:47.323343 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4250/8429] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:47.327581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.337352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3252cf-d1b6-48e9-accc-dbd2d13dcb8f None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.341914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.342568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.342852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.343123 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.343765 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:47.344068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.345665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.357074 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4180/8430] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:29:47.361461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.371372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc95-039e-427e-adbf-419fe1daacc3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.376203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.376798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.377076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.377346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.378000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.380716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.409027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4251/8431] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:47.413244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.423682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fa856-00d7-4f1c-90a0-968b2352caf1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.479250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.479781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.480060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.480334 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.480963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.482932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.504765 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4181/8432] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:47.509607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.519966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58c19b-4bef-422e-94bb-3f12235d2950 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.524686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.525527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.525801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.526066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.526722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.528286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.538106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4252/8433] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:47.542001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.552482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a860663-ff5e-407a-8e01-e532f4e59793 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.557141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.557724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.558000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.558270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.558942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.560522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceac249-c85d-48ad-bc88-c5f2169e1a11 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.571912 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4182/8434] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:47.575900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.585977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae04c2-85e0-48bc-8892-8c1bcfc1a4c9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.590661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.591305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.591585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.591858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] PATH_INFO: `/v3/projects/6ac6c95d735f47b1b15a43dc2b127895/users/33d15a7b4f924b59881ad41b92b8b708/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.602835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:47.607490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac6c95d735f47b1b15a43dc2b127895, user_id=33d15a7b4f924b59881ad41b92b8b708, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.610938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.625091 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4253/8435] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:47 2022] PUT /identity/v3/projects/6ac6c95d735f47b1b15a43dc2b127895/users/33d15a7b4f924b59881ad41b92b8b708/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:47.636297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.636297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.636546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.664576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] MFA Rules not processed for user `33d15a7b4f924b59881ad41b92b8b708`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:47.665806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:47.701361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:47.734047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4183/8436] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:47.738441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.799985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0369e-117e-437a-baa4-8f22e719c6bd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.804939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.805541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.805821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.806094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.806787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.808740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.838448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4254/8437] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:47.839701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.850567 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c04b13-4c85-4855-b820-d1422530c4b7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.906214 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.906853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.907135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.907409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.908032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.909952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.931856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4184/8438] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:47 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:47.935824 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.946905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c73f9-490b-4443-b4aa-7f2d6fd0ee91 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.951492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.952076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.952350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.952617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:47.953243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:47.954873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:47.967381 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4255/8439] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:47.971398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:47.981853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce033-fc72-4641-bb05-93293613bce1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:47.986663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:47.987255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:47.987537 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:47.987817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] PATH_INFO: `/v3/projects/b59122b01f614be2ad3808d19ee05394/users/59845e86bfa64510a3b4b81264e187df/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.002638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59122b01f614be2ad3808d19ee05394, user_id=59845e86bfa64510a3b4b81264e187df, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:48.005994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:48.019462 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4185/8440] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:47 2022] PUT /identity/v3/projects/b59122b01f614be2ad3808d19ee05394/users/59845e86bfa64510a3b4b81264e187df/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:48.023990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:48.034412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd241a8-16a7-4507-8671-96387755cd12 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:48.090279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:48.091218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:48.091466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:48.091738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.092365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:48.093947 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cbcbfe-014b-44dd-aad1-29e0a79136fc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:48.103796 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4256/8441] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:48.107883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:48.119097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecba47-4551-47c9-9467-9fa04c94ea40 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:48.123793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:48.124435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:48.124714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:48.124986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] PATH_INFO: `/v3/projects/b59122b01f614be2ad3808d19ee05394/users/59845e86bfa64510a3b4b81264e187df/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.126958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59122b01f614be2ad3808d19ee05394, user_id=59845e86bfa64510a3b4b81264e187df, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:48.129585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:48.140184 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4186/8442] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:48 2022] PUT /identity/v3/projects/b59122b01f614be2ad3808d19ee05394/users/59845e86bfa64510a3b4b81264e187df/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:48.151617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:48.152005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:48.152385 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.180156 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] MFA Rules not processed for user `59845e86bfa64510a3b4b81264e187df`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:48.181349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:48.227581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:48.250642 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4257/8443] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:48.261822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:48.263544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:48.336709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e72258-b679-47ca-af0c-6579d073cb63 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:48.340345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:48.340931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:48.341208 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:48.341477 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.351223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:48.352661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf819-749d-4e18-9e3f-02fc37c56238 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:48.362477 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4187/8444] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:48.908293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:48.972885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3f84e-9bc2-4373-8e5a-ff52b9fba231 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:48.978201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:48.978854 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:48.979129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:48.979397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] PATH_INFO: `/v3/users/33d15a7b4f924b59881ad41b92b8b708` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:48.980515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d15a7b4f924b59881ad41b92b8b708)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:48.982429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:49.028965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d15a7b4f924b59881ad41b92b8b708'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.029612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d15a7b4f924b59881ad41b92b8b708'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.039048 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d15a7b4f924b59881ad41b92b8b708'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.051199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d15a7b4f924b59881ad41b92b8b708'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.052044 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d15a7b4f924b59881ad41b92b8b708'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.064107 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4258/8445] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:48 2022] DELETE /identity/v3/users/33d15a7b4f924b59881ad41b92b8b708 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:49.068408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:49.068694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:49.068965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:49.089067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:49.090270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:49.103320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:49.149560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4188/8446] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:49.159510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:49.160898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:49.219986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18635f1-ea98-4150-8b44-544853742fe7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:49.223635 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:49.224343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:49.224628 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:49.225179 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:49.234062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:49.235507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:49.244911 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4259/8447] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:49.511345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:49.512823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:49.526375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0f5c7-b17e-4575-8ac7-3590c731158e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:49.531118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:49.531746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:49.532020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:49.532286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] PATH_INFO: `/v3/projects/6ac6c95d735f47b1b15a43dc2b127895` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:49.533458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac6c95d735f47b1b15a43dc2b127895)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:49.535332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:49.582375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac6c95d735f47b1b15a43dc2b127895', 'request_id': 'req-7775e0fc-162c-41a6-9158-c041aaae856f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.588023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac6c95d735f47b1b15a43dc2b127895', 'request_id': 'req-7775e0fc-162c-41a6-9158-c041aaae856f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.595790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac6c95d735f47b1b15a43dc2b127895', 'request_id': 'req-7775e0fc-162c-41a6-9158-c041aaae856f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.606813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] The token cache is being invalidate because project 6ac6c95d735f47b1b15a43dc2b127895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:49.607087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.607710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.609185 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4189/8448] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:49 2022] DELETE /identity/v3/projects/6ac6c95d735f47b1b15a43dc2b127895 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:49.613113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:49.623602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311809cf-960a-4cc3-9b9e-36febbc03efc service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:49.627169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:49.689569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:49.690226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:49.690514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:49.690807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] PATH_INFO: `/v3/users/59845e86bfa64510a3b4b81264e187df` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:49.691912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59845e86bfa64510a3b4b81264e187df)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:49.694874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:49.740475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59845e86bfa64510a3b4b81264e187df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.741129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59845e86bfa64510a3b4b81264e187df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.748387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59845e86bfa64510a3b4b81264e187df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.761069 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59845e86bfa64510a3b4b81264e187df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.761721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-cdef139f-41c2-41cc-a4c4-2ed66180357b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59845e86bfa64510a3b4b81264e187df'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:49.773924 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4260/8449] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:49 2022] DELETE /identity/v3/users/59845e86bfa64510a3b4b81264e187df => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:50.064436 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.065906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:50.079997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775e0fc-162c-41a6-9158-c041aaae856f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.138790 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.139351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.139631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.139921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] PATH_INFO: `/v3/projects/b59122b01f614be2ad3808d19ee05394` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.141069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59122b01f614be2ad3808d19ee05394)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.143042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.186989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:50.190227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59122b01f614be2ad3808d19ee05394', 'request_id': 'req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:50.196597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59122b01f614be2ad3808d19ee05394', 'request_id': 'req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:50.204077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59122b01f614be2ad3808d19ee05394', 'request_id': 'req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:50.215695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] The token cache is being invalidate because project b59122b01f614be2ad3808d19ee05394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:50.215982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:50.216617 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-caac1c03-4a54-4dee-8cb2-fbf13fe6341f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:50.218105 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4190/8450] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:50 2022] DELETE /identity/v3/projects/b59122b01f614be2ad3808d19ee05394 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:50.223269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.224217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.224431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.246375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:50.247616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:50.306332 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4261/8451] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:50.314182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.314467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.314773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.335089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:50.336399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:50.344456 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4191/8452] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:50.348431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.358935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd887a-34b0-46db-87f9-ba92d421814b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.363532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.364123 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.364721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.364999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.365637 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:50.365940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.367585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.380015 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4262/8453] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:50.383224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.392954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c77d-12ad-4fab-8508-c627141d8bc4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.398228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.398864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.399170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.399445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.400077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.402237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.430552 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4192/8454] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:50 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:50.434629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.444932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d73da-6a64-459f-b43a-94f32a5e2305 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.454034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:50.505221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.506227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.506596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.506862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.507491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.509388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.531011 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4263/8455] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:50 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:50.535106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.546430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7cc3-5b93-487e-9ab0-61ecadb228ef admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.551044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.551629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.552023 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.552504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.553261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.554888 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.564560 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4193/8456] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:50.568478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.578608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d91fdc-79c3-468e-bb43-73dc30f28352 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.583916 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.584501 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.584784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.585061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.585711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.587325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b5a30-3654-462c-98c7-9d19b706891e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.597775 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4264/8457] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:50.602358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.612185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb5d3c-6fa2-4727-9d22-766e27df020a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.617126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.617757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.618032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.618305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] PATH_INFO: `/v3/projects/d77d4111899f47ae88f07857ef03d1ba/users/8e99c2149382424481706b0f02b627ac/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.633074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77d4111899f47ae88f07857ef03d1ba, user_id=8e99c2149382424481706b0f02b627ac, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.635767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.650065 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4194/8458] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:50 2022] PUT /identity/v3/projects/d77d4111899f47ae88f07857ef03d1ba/users/8e99c2149382424481706b0f02b627ac/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:29:50.660551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.660843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.661120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.689999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] MFA Rules not processed for user `8e99c2149382424481706b0f02b627ac`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:50.691284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:50.758234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4265/8459] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:50.768912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:50.771448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:50.830545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9dcbd-ac0a-4452-a40b-04c3a1e7fb77 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:50.835280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:50.835871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:50.836146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:50.836415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:50.844756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:50.846188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41073cc8-a37e-4a68-a09c-8f22b4c14037 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:50.855306 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4195/8460] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:52.335876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:52.398700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e2aaf-9639-4154-afc0-3a10024c6e59 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:52.403468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:52.404140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:52.404363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:52.404634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] PATH_INFO: `/v3/users/8e99c2149382424481706b0f02b627ac` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:52.405747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e99c2149382424481706b0f02b627ac)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:52.407684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:52.429292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:52.454822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e99c2149382424481706b0f02b627ac'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:52.455470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e99c2149382424481706b0f02b627ac'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:52.463505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e99c2149382424481706b0f02b627ac'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:52.475476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e99c2149382424481706b0f02b627ac'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:52.476102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e99c2149382424481706b0f02b627ac'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:52.488043 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4266/8461] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:52 2022] DELETE /identity/v3/users/8e99c2149382424481706b0f02b627ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:52.493352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:52.493799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:52.493883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:52.513071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:52.514255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:52.574030 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4196/8462] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:52.583979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:52.585388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:52.645492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec549e0a-bc29-49bb-b639-d581f54dd773 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:52.649554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:52.650229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:52.650810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:52.651079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:52.660245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:52.661837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532e93-5d80-452d-a685-3d7281eb9c5e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:52.671556 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4267/8463] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:52.944904 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:52.946519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:52.965149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925bf62-7fb4-4478-8969-cc2a9e6e702d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:52.969915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:52.970559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:52.970842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:52.971114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] PATH_INFO: `/v3/projects/d77d4111899f47ae88f07857ef03d1ba` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:52.972682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77d4111899f47ae88f07857ef03d1ba)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:52.975267 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:52.984479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:53.024242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77d4111899f47ae88f07857ef03d1ba', 'request_id': 'req-2bb573a2-ca78-469e-9e0e-671ff0704ed1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:53.030231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77d4111899f47ae88f07857ef03d1ba', 'request_id': 'req-2bb573a2-ca78-469e-9e0e-671ff0704ed1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:53.037381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77d4111899f47ae88f07857ef03d1ba', 'request_id': 'req-2bb573a2-ca78-469e-9e0e-671ff0704ed1'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:53.048755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] The token cache is being invalidate because project d77d4111899f47ae88f07857ef03d1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:53.049046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:53.049686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2bb573a2-ca78-469e-9e0e-671ff0704ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:53.051208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4197/8464] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:52 2022] DELETE /identity/v3/projects/d77d4111899f47ae88f07857ef03d1ba => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:53.059298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.059587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.059860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.079454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:53.080671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:53.140254 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4268/8465] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:53.149167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.149455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.149728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.170942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:53.172163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:53.180682 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4198/8466] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:53.184791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.194838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b232e4-2652-41ed-bdcb-30396567ae34 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.199503 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.200096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.200378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.200653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.201295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:53.201600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.203266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.214679 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4269/8467] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:53.218762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.228780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262136e-2082-41ff-b629-f9ad7ce71810 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.233365 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.233955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.234233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.234528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.235144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.237059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.263827 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4199/8468] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:53 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:53.268087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.279891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172f910-cbd2-4f70-ad5b-6634ad48e386 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.312669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:53.345752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.346481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.346660 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.346934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.347565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.350170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.371328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4270/8469] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:53 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:53.375407 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.385875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8660175-3c73-408b-aa1d-c0e1298de95b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.390540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.391130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.391403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.391673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.392299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.393897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.405032 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4200/8470] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:53.408815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.419481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d250f0-89be-434d-bcdf-1ec56b906ded admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.424557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.425308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.425474 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.425894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.426598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.428239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f856ec4-a0f7-4bcd-abc3-5b8d45d7f2c2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.438266 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4271/8471] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:29:53.442509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.453061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d2771-4f69-4b97-8232-8345cd48924f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.457685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.458865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.459138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.459480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] PATH_INFO: `/v3/projects/6a601c93303246f5a5311aecdf5f2d50/users/e8df5dd8896145f687b2e36c9d1fdfe5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.474581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a601c93303246f5a5311aecdf5f2d50, user_id=e8df5dd8896145f687b2e36c9d1fdfe5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.477318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.490399 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4201/8472] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:53 2022] PUT /identity/v3/projects/6a601c93303246f5a5311aecdf5f2d50/users/e8df5dd8896145f687b2e36c9d1fdfe5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:53.500557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.500843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.501116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.530155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] MFA Rules not processed for user `e8df5dd8896145f687b2e36c9d1fdfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:53.531403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:53.595828 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4272/8473] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:53.608220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:53.609869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:53.669756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723bdea-e32d-4445-a04a-d5ac84fd63a1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:53.673430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:53.674027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:53.674308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:53.674613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:53.683704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:53.685165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54d7a-4d01-4c4b-83e1-fd0f00449938 service cinder] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:53.694580 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4202/8474] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:55.185662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:55.248279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd289fb-908c-45e1-a095-adb4c707b8eb None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:55.252869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:55.253550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:55.253783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:55.254051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] PATH_INFO: `/v3/users/e8df5dd8896145f687b2e36c9d1fdfe5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:55.255238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8df5dd8896145f687b2e36c9d1fdfe5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:55.257148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:55.297656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:55.303948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8df5dd8896145f687b2e36c9d1fdfe5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.304846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8df5dd8896145f687b2e36c9d1fdfe5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.311950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8df5dd8896145f687b2e36c9d1fdfe5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.323716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8df5dd8896145f687b2e36c9d1fdfe5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.324359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8df5dd8896145f687b2e36c9d1fdfe5'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.336373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4273/8475] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:29:55 2022] DELETE /identity/v3/users/e8df5dd8896145f687b2e36c9d1fdfe5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:55.340948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:55.341409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:55.341495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:55.361313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:55.362539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:55.394833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:55.431464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4203/8476] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:29:55.441251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:55.442723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:55.503122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3775b-1cfb-4706-945e-c97d1568f646 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:55.506986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:55.507526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:55.507812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:55.508086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:55.517144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:55.518676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e333f4-a214-435b-8899-71cb546e9978 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:55.528405 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4274/8477] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:29:55.804009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:55.805444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:55.819933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45b259-05d8-4d0c-8335-c54b34614898 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:55.833064 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:55.834388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:55.834761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:55.835101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] PATH_INFO: `/v3/projects/6a601c93303246f5a5311aecdf5f2d50` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:55.836297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a601c93303246f5a5311aecdf5f2d50)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:55.838298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:55.891535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a601c93303246f5a5311aecdf5f2d50', 'request_id': 'req-74d23dde-094a-4450-b7bb-f9e6da02ec34'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.897244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a601c93303246f5a5311aecdf5f2d50', 'request_id': 'req-74d23dde-094a-4450-b7bb-f9e6da02ec34'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.904683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a601c93303246f5a5311aecdf5f2d50', 'request_id': 'req-74d23dde-094a-4450-b7bb-f9e6da02ec34'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.916629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] The token cache is being invalidate because project 6a601c93303246f5a5311aecdf5f2d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:29:55.916946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.917610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-74d23dde-094a-4450-b7bb-f9e6da02ec34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:29:55.919140 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4204/8478] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:29:55 2022] DELETE /identity/v3/projects/6a601c93303246f5a5311aecdf5f2d50 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:55.923951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:55.924235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:55.924505 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:55.944870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:55.946074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:56.005262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4275/8479] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:56.013012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.013305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.013584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.033259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:56.034457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:56.043366 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4205/8480] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:56.047560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.058331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9a712-9644-439d-94a8-65dd2e8973d4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.062914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.063515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.063794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.064068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.064704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:29:56.065008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.066631 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.079377 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4276/8481] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:29:56.083700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.093015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d2cf1-c49e-42c7-bfa7-96885a526475 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.097685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.098273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.098589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.098869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.100168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.102081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.129992 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4206/8482] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:56.134333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.144472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd44ce-86b1-4306-a48c-7cb0df795133 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.189708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:56.203184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.203809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.204092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.204368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.205014 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.206962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.231061 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4277/8483] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:29:56.235303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.246451 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38335-9e5d-430a-a522-18b90246354f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.251000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.251592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.251885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.252136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.252762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.254386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.264329 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4207/8484] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:56.268239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.278508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ce9df-e970-415a-be6e-943408c3109b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.283217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.283811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.284085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.284355 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.284979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.287152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839b46-ba4c-46b6-a0ab-583d93f486a3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.297940 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4278/8485] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:56.301898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.312449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da87d-4605-4d15-ba61-fa9f963eb984 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.317076 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.317718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.318010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.318286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] PATH_INFO: `/v3/projects/51a1a71cc0014bfea7a53f101c0fd196/users/674a602d792641a0b5f9b4aefd73303a/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.334807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a1a71cc0014bfea7a53f101c0fd196, user_id=674a602d792641a0b5f9b4aefd73303a, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.337492 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.351147 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4208/8486] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:56 2022] PUT /identity/v3/projects/51a1a71cc0014bfea7a53f101c0fd196/users/674a602d792641a0b5f9b4aefd73303a/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:29:56.362232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.362541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.362825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.390231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] MFA Rules not processed for user `674a602d792641a0b5f9b4aefd73303a`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:56.391471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:56.455379 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4279/8487] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:56.460445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.516946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:56.523643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19e601-89ea-4266-aa85-478327ed18ea admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.529480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.530078 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.530387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.530701 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.531340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.533256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.562601 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4209/8488] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:29:56.563817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.575389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b5d66-2e7a-4775-89a8-2d577f54b21d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.633369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.634026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.634302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.634630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.635265 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.637589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.659688 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4280/8489] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:56.664388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.674770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e33a5-9bcd-4d49-9947-9128ad197455 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.680081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.680677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.680957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.681228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.681854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.683487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.694784 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4210/8490] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:56.698853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.709538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3eac-749d-4545-a17a-b6c0f6c2ecb3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.714083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.714723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.714998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.715264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.715890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.717451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cab68-4b27-42d4-9fb9-5bd95c33d5fb admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.728031 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4281/8491] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:56.731649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.744929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a9426-2b01-438a-b8dd-f7c164a7383f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.749587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.750264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.750579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.750858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] PATH_INFO: `/v3/projects/846853bc288e4a728b37e8dc09a2c98b/users/7a15c62e64704bab9eb1def03eb0718d/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.766956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=846853bc288e4a728b37e8dc09a2c98b, user_id=7a15c62e64704bab9eb1def03eb0718d, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.769632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.783757 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4211/8492] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:56 2022] PUT /identity/v3/projects/846853bc288e4a728b37e8dc09a2c98b/users/7a15c62e64704bab9eb1def03eb0718d/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:56.794648 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313408-fde7-498c-894e-712487ced48a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.794937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313408-fde7-498c-894e-712487ced48a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.795207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313408-fde7-498c-894e-712487ced48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.823015 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-9a313408-fde7-498c-894e-712487ced48a None None] MFA Rules not processed for user `7a15c62e64704bab9eb1def03eb0718d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:56.824141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-9a313408-fde7-498c-894e-712487ced48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:56.848072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:56.888368 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4282/8493] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:56.892967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:56.955592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf7bd0-1cbd-4c4a-86e9-8a0e6c04df8e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:56.960885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:56.961521 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:56.961801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:56.962072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:56.962739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:56.964650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:56.995858 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4212/8494] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:56.997034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a313408-fde7-498c-894e-712487ced48a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.007816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a313408-fde7-498c-894e-712487ced48a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.064639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.065241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.065518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.065786 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.066402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.068657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.090180 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4283/8495] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:29:56 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:29:57.094183 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.105250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27050657-268b-41bf-a183-cf0144f1ecb7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.109823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.110411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.110726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.110998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.111624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.113211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.123644 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4213/8496] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:57.127560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.137817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b005e-94ef-497a-8505-2b84840749bc admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.142486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.143155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.143434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.143706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] PATH_INFO: `/v3/projects/1a0b200e0bb447529d3b02cd729aec47/users/8e3a2c0615524a39ac9a6a0a163edee6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.159812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0b200e0bb447529d3b02cd729aec47, user_id=8e3a2c0615524a39ac9a6a0a163edee6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.163241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.175941 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4284/8497] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:57 2022] PUT /identity/v3/projects/1a0b200e0bb447529d3b02cd729aec47/users/8e3a2c0615524a39ac9a6a0a163edee6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:29:57.180038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.190466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b8689-77d5-4084-9e88-a6d5dc8e84ba admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.246575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.247170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.247446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.247716 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.248337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.249957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d9e7c-6cab-4703-9932-cd23ca2cda21 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.259897 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4214/8498] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:29:57.263968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.274727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881733a-03cb-499c-8e23-172a15207b82 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.279567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.280201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.280477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.280751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] PATH_INFO: `/v3/projects/1a0b200e0bb447529d3b02cd729aec47/users/8e3a2c0615524a39ac9a6a0a163edee6/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.282675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0b200e0bb447529d3b02cd729aec47, user_id=8e3a2c0615524a39ac9a6a0a163edee6, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.285373 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.296635 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4285/8499] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:29:57 2022] PUT /identity/v3/projects/1a0b200e0bb447529d3b02cd729aec47/users/8e3a2c0615524a39ac9a6a0a163edee6/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:29:57.307868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.308155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.308458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.336131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] MFA Rules not processed for user `8e3a2c0615524a39ac9a6a0a163edee6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:29:57.337331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:57.342435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:57.409476 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4215/8500] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:29:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:29:57.421917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:29:57.423768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:29:57.484956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202efcbc-b759-4be1-98aa-e91756531880 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:29:57.488644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:29:57.489242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:29:57.489622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:29:57.489971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:29:57.520017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:29:57.548196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:29:57.549616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0e481-1fe3-4ff2-b811-e98137c8fd04 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:29:57.558637 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4286/8501] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:29:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:00.502116 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:00.563858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5134b-bf3e-4ace-80a7-ff2a36a5b95c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:00.569012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:00.569637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:00.569938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:00.570213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] PATH_INFO: `/v3/users/674a602d792641a0b5f9b4aefd73303a` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:00.571377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=674a602d792641a0b5f9b4aefd73303a)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:00.573308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:00.617955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674a602d792641a0b5f9b4aefd73303a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:00.618700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674a602d792641a0b5f9b4aefd73303a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:00.625807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674a602d792641a0b5f9b4aefd73303a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:00.633639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674a602d792641a0b5f9b4aefd73303a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:00.634303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674a602d792641a0b5f9b4aefd73303a'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:00.653008 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4216/8502] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:00 2022] DELETE /identity/v3/users/674a602d792641a0b5f9b4aefd73303a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:00.657470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:00.657902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:00.658018 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:00.677751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:00.679083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:00.738235 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4287/8503] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:00.748919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:00.750322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:00.772795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:00.812698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3cbd9-fe97-4f87-aaa5-bf9ea0f70a7d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:00.816341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:00.816929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:00.817204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:00.817472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:00.827361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:00.828794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:00.838382 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4217/8504] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:01.106558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:01.108004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:01.120728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f239d-885e-4947-8cb7-38eb25ec081c None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:01.125330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:01.126699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:01.126980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:01.127587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] PATH_INFO: `/v3/projects/51a1a71cc0014bfea7a53f101c0fd196` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:01.128730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a1a71cc0014bfea7a53f101c0fd196)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:01.131525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:01.183827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a1a71cc0014bfea7a53f101c0fd196', 'request_id': 'req-b99d00e8-1e7b-4ced-a276-9937439b75ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.189158 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a1a71cc0014bfea7a53f101c0fd196', 'request_id': 'req-b99d00e8-1e7b-4ced-a276-9937439b75ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.195730 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a1a71cc0014bfea7a53f101c0fd196', 'request_id': 'req-b99d00e8-1e7b-4ced-a276-9937439b75ff'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.207915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] The token cache is being invalidate because project 51a1a71cc0014bfea7a53f101c0fd196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:01.208207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.209810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.211438 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4288/8505] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:01 2022] DELETE /identity/v3/projects/51a1a71cc0014bfea7a53f101c0fd196 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:01.215597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:01.225469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92f5d9-3cba-405c-9776-25f191cfc2dc service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:01.281931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:01.282586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:01.282890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:01.283156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] PATH_INFO: `/v3/users/7a15c62e64704bab9eb1def03eb0718d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:01.284266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a15c62e64704bab9eb1def03eb0718d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:01.286237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:01.330219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a15c62e64704bab9eb1def03eb0718d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.330878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a15c62e64704bab9eb1def03eb0718d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.338706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a15c62e64704bab9eb1def03eb0718d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.345430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a15c62e64704bab9eb1def03eb0718d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.346069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a15c62e64704bab9eb1def03eb0718d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.362898 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4218/8506] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:01 2022] DELETE /identity/v3/users/7a15c62e64704bab9eb1def03eb0718d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:01.642939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:01.644462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:01.662075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d00e8-1e7b-4ced-a276-9937439b75ff admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:01.712471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:01.720352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:01.721032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:01.721510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:01.721802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] PATH_INFO: `/v3/projects/846853bc288e4a728b37e8dc09a2c98b` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:01.723260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846853bc288e4a728b37e8dc09a2c98b)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:01.725279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:01.772404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846853bc288e4a728b37e8dc09a2c98b', 'request_id': 'req-050fbe83-5d71-4586-85b4-6f62fd81fe17'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.778119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846853bc288e4a728b37e8dc09a2c98b', 'request_id': 'req-050fbe83-5d71-4586-85b4-6f62fd81fe17'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.784199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846853bc288e4a728b37e8dc09a2c98b', 'request_id': 'req-050fbe83-5d71-4586-85b4-6f62fd81fe17'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.796325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] The token cache is being invalidate because project 846853bc288e4a728b37e8dc09a2c98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:01.796615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.797230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.798731 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4289/8507] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:01 2022] DELETE /identity/v3/projects/846853bc288e4a728b37e8dc09a2c98b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:01.803394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:01.804843 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:01.818536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2c22-a125-463c-aa0b-dd95bfad20a4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:01.858905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:01.877540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:01.878174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:01.878450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:01.878754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] PATH_INFO: `/v3/users/8e3a2c0615524a39ac9a6a0a163edee6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:01.879869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3a2c0615524a39ac9a6a0a163edee6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:01.882645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:01.930855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3a2c0615524a39ac9a6a0a163edee6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.931502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3a2c0615524a39ac9a6a0a163edee6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.938228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3a2c0615524a39ac9a6a0a163edee6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.945198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3a2c0615524a39ac9a6a0a163edee6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.945870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e73e92b1-e2e2-45e6-8c93-b432295ff77f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3a2c0615524a39ac9a6a0a163edee6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:01.962876 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4219/8508] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:01 2022] DELETE /identity/v3/users/8e3a2c0615524a39ac9a6a0a163edee6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:02.243490 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.245170 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:02.258557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fbe83-5d71-4586-85b4-6f62fd81fe17 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.325165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.325794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.326070 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.326343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] PATH_INFO: `/v3/projects/1a0b200e0bb447529d3b02cd729aec47` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.327467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0b200e0bb447529d3b02cd729aec47)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.329348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.375376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0b200e0bb447529d3b02cd729aec47', 'request_id': 'req-e0018d35-859b-47c3-8236-b47e01a4bbca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:02.380843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0b200e0bb447529d3b02cd729aec47', 'request_id': 'req-e0018d35-859b-47c3-8236-b47e01a4bbca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:02.386804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0b200e0bb447529d3b02cd729aec47', 'request_id': 'req-e0018d35-859b-47c3-8236-b47e01a4bbca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:02.399638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] The token cache is being invalidate because project 1a0b200e0bb447529d3b02cd729aec47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:02.399924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:02.400588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e0018d35-859b-47c3-8236-b47e01a4bbca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:02.402351 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4290/8509] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:02 2022] DELETE /identity/v3/projects/1a0b200e0bb447529d3b02cd729aec47 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:02.407228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.407510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.407781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.428216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:02.429408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:02.489094 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4220/8510] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:02.497004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.497293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.497569 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.517766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:02.518991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:02.527645 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4291/8511] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:02.531776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.542300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9ee2b-8144-4373-839b-8857f86ee83e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.546899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.547486 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.547760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.548025 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.548659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:30:02.548954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.550556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.561814 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4221/8512] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:30:02.566029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.577257 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb2060-29e6-4fb6-bb98-709f5f581b4d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.581881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.582464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.582814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.583081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.584212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.586196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.591435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:02.614008 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4292/8513] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:02 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:02.618250 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.628302 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a630cb2-dd1f-44aa-a56c-9c39700d4aac admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.675062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:02.690185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.690831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.691115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.691392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.692045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.694705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.716143 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4222/8514] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:02 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:02.720278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.730749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9d442-f5b6-441f-bb51-9d4f4d921b69 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.749207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.750102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.752495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.752880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.753754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.758041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.775611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4293/8515] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:02.778930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.789788 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228861e-2941-4147-9304-89c3fc6616f7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.795228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.795932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.796290 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.796618 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.797287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.798926 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1cb25-b6d0-4ffe-bcf6-e343c5e8903e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.810332 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4223/8516] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:30:02.814670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.825861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0e6df-858e-49a4-834c-bc697ed6933c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:02.830633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:02.831318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.831559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.831839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] PATH_INFO: `/v3/projects/3c03ef0648f545a8aa6e450457ddb489/users/736d77742bbe428ab07e8b228c9ee7f5/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.847626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c03ef0648f545a8aa6e450457ddb489, user_id=736d77742bbe428ab07e8b228c9ee7f5, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:02.850517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:02.863811 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4294/8517] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:02 2022] PUT /identity/v3/projects/3c03ef0648f545a8aa6e450457ddb489/users/736d77742bbe428ab07e8b228c9ee7f5/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:02.874197 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:02.874540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:02.874821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:02.904009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] MFA Rules not processed for user `736d77742bbe428ab07e8b228c9ee7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:02.905225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:02.971391 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4224/8518] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:02.984147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:02.985867 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:03.047089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2763c-c5f4-4d49-88e4-946ff922fcf1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:03.050792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:03.051393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:03.051676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:03.051951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:03.061079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:03.062540 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f0edd-5778-4df5-affa-35a6808554fd service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:03.072211 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4295/8519] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:06.153616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:06.215178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccff85-5fe2-4cb8-b35a-c9296be82417 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:06.219766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:06.220387 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.220671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.220947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] PATH_INFO: `/v3/users/736d77742bbe428ab07e8b228c9ee7f5` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.222058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=736d77742bbe428ab07e8b228c9ee7f5)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:06.224222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:06.270196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736d77742bbe428ab07e8b228c9ee7f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.270871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736d77742bbe428ab07e8b228c9ee7f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.278873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736d77742bbe428ab07e8b228c9ee7f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.283395 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:06.288664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736d77742bbe428ab07e8b228c9ee7f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.289569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736d77742bbe428ab07e8b228c9ee7f5'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.307875 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4225/8520] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:06 2022] DELETE /identity/v3/users/736d77742bbe428ab07e8b228c9ee7f5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:06.312108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.312594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.312687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.336967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:06.338183 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:06.395268 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4296/8521] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:06.405786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:06.407236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:06.466939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d100-2e9c-49fb-8048-fef5bed78908 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:06.470596 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:06.471196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.471473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.471745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.480900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:06.482335 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a87c-bc93-476e-9d6c-6b0d678a3427 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:06.491934 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4226/8522] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:06.774207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:06.775684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:06.788938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f613613-1ede-460c-99a7-64cf173dd2d8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:06.793676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:06.794378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.794762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.795105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] PATH_INFO: `/v3/projects/3c03ef0648f545a8aa6e450457ddb489` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.796364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c03ef0648f545a8aa6e450457ddb489)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:06.798280 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:06.846603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c03ef0648f545a8aa6e450457ddb489', 'request_id': 'req-2864eb5f-a080-4221-80b7-a8581a7669ca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.847777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:06.854912 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c03ef0648f545a8aa6e450457ddb489', 'request_id': 'req-2864eb5f-a080-4221-80b7-a8581a7669ca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.861560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c03ef0648f545a8aa6e450457ddb489', 'request_id': 'req-2864eb5f-a080-4221-80b7-a8581a7669ca'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.874669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] The token cache is being invalidate because project 3c03ef0648f545a8aa6e450457ddb489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:06.874963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.875592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2864eb5f-a080-4221-80b7-a8581a7669ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:06.877087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4297/8523] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:06 2022] DELETE /identity/v3/projects/3c03ef0648f545a8aa6e450457ddb489 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:06.882176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.882461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.882771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.903254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:06.904398 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:06.962899 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4227/8524] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:06.971601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:06.971891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:06.972166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:06.992457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:06.993655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:07.001515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4298/8525] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:07.006047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.016112 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb38cb-66b3-41cc-937b-3b1d0601b30d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.020853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.022182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.022587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.022863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.023500 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:30:07.023803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.026044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.038463 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4228/8526] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:30:07.042986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.052242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb04f9a-76d6-42a9-ad46-832ed4cf981e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.056935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.057524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.057799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.058068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.058740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.060626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.087712 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4299/8527] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:07 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:07.091918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.102480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6518b8-647e-4f97-8a04-f63f4d44fe6b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.160829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.161446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.161604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.162524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.163159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.165107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.176118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:07.188005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4229/8528] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:07 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:30:07.192117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.203282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1274f9-7b7a-4ead-b572-ec26c4be4518 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.207940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.208533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.209377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.209649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.210278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.211909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.221532 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4300/8529] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:07.225359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.235383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686434e-17b8-4c53-8dec-5a8e0b4ea463 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.240126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.240810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.241165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.241467 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.242115 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.243733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d8001-ce8c-44fa-bce4-1bc23f4db823 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.254210 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4230/8530] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:30:07.258869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.268738 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4618f9-19c4-49f2-b337-9851392557b9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.274169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.274851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.275129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.275404 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] PATH_INFO: `/v3/projects/cb93f40253554a6ea7d966d136bf3f23/users/f24dbc0c130647afa5692569082a41f4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.291627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb93f40253554a6ea7d966d136bf3f23, user_id=f24dbc0c130647afa5692569082a41f4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.294321 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.307403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4301/8531] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:07 2022] PUT /identity/v3/projects/cb93f40253554a6ea7d966d136bf3f23/users/f24dbc0c130647afa5692569082a41f4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:30:07.317669 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.318478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.318872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.347260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] MFA Rules not processed for user `f24dbc0c130647afa5692569082a41f4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:07.348489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:07.415083 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4231/8532] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:07.426679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:07.428383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:07.488159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c313302-1200-4dd7-b4be-1e0d33df4c13 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:07.492358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:07.492977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:07.493251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:07.493548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:07.501743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:07.503256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a73430-2dd1-4293-a864-47ba65c92e1d service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:07.512976 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4302/8533] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:10.608885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:10.674556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fbaeb-a018-48b4-8fec-a90de71ff27d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:10.680658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:10.681284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:10.681563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:10.681857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] PATH_INFO: `/v3/users/f24dbc0c130647afa5692569082a41f4` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:10.683005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f24dbc0c130647afa5692569082a41f4)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:10.684956 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:10.731220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f24dbc0c130647afa5692569082a41f4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:10.731863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f24dbc0c130647afa5692569082a41f4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:10.739082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f24dbc0c130647afa5692569082a41f4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:10.747008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f24dbc0c130647afa5692569082a41f4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:10.747653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f24dbc0c130647afa5692569082a41f4'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:10.758432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:10.766882 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4232/8534] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:10 2022] DELETE /identity/v3/users/f24dbc0c130647afa5692569082a41f4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:10.771384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:10.771980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:10.772077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:10.791140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:10.792354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:10.851447 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4303/8535] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:10.861122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:10.862572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:10.925014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a580f-7f29-44ef-b2d1-027f07591f63 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:10.930313 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:10.930993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:10.931270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:10.931538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:10.940646 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:10.942072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c99ffb-3d87-4b6d-a831-d7d16ae5230f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:10.951818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4233/8536] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:11.219111 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.220518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:11.235451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac3b14-c149-4414-b965-e7eada8693b8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.241676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.242791 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.243061 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.243328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] PATH_INFO: `/v3/projects/cb93f40253554a6ea7d966d136bf3f23` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.244582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb93f40253554a6ea7d966d136bf3f23)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.246929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.259211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:11.300455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb93f40253554a6ea7d966d136bf3f23', 'request_id': 'req-0d4bcc95-63da-440c-bd32-31a2a2076c7e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:11.306985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb93f40253554a6ea7d966d136bf3f23', 'request_id': 'req-0d4bcc95-63da-440c-bd32-31a2a2076c7e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:11.313140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb93f40253554a6ea7d966d136bf3f23', 'request_id': 'req-0d4bcc95-63da-440c-bd32-31a2a2076c7e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:11.325700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] The token cache is being invalidate because project cb93f40253554a6ea7d966d136bf3f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:11.325992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:11.326661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-0d4bcc95-63da-440c-bd32-31a2a2076c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:11.328163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4304/8537] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:11 2022] DELETE /identity/v3/projects/cb93f40253554a6ea7d966d136bf3f23 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:11.333024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.333348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.333623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.354566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:11.355720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:11.414470 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4234/8538] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:11.422382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.422696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.422967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.443333 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:11.444588 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:11.452712 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4305/8539] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:11.456856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.467127 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d5c4c-7d01-4acf-851c-8ca91eb616fd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.471999 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.472630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.472917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.473187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.473831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:30:11.474133 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.476704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.488165 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4235/8540] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:30:11.492482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.501739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f99e-6c85-49a7-8b14-ffd38669e34a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.506435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.507049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.507324 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.507592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.508234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.510114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.538025 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4306/8541] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:11 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:11.542137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.552481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd80d23-e6c2-4fa9-bde2-e62e65f80e7f admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.608996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.609591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.609815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.610086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.610771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.612710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.633240 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4236/8542] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:11 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:11.637498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.648794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94ca98-26a2-4baf-828e-d9a16ae3eb62 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.653675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.654275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.654581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.654856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.655492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.657122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.667682 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4307/8543] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:30:11.671686 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.682877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f2ab2-cbc9-4d13-a644-55c4a315fbd2 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.687604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.688196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.688478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.688751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.689412 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.691072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c3552-d948-4f8a-b8c8-98766f42406b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.702402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4237/8544] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:11.706626 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.717592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0992176-766e-473f-89e8-e819d921294c admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.722161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.722829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.723106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.723375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] PATH_INFO: `/v3/projects/a25a574ff0384f89b15634566b0a82b6/users/b0a0fbd606f34d589c744a8949c7f12f/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.737712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25a574ff0384f89b15634566b0a82b6, user_id=b0a0fbd606f34d589c744a8949c7f12f, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.740646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.753213 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4308/8545] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:11 2022] PUT /identity/v3/projects/a25a574ff0384f89b15634566b0a82b6/users/b0a0fbd606f34d589c744a8949c7f12f/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:11.765080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.765366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.765639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.773375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:11.795426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] MFA Rules not processed for user `b0a0fbd606f34d589c744a8949c7f12f`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:11.796613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:11.860322 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4238/8546] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:11.871971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:11.873662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:11.933293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106b9ce-48a2-439b-a5a3-601a26313698 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:11.936965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66ad10d7496429da3b81648cf9e83f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:11.937560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:11.937842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:11.938129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:11.947003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:11.948422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0aa49-979f-4cb6-b3ad-01f1992d4e98 service cinder] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:11.957142 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4309/8547] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:13.618734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:13.686292 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad011e0-8cfa-44d5-8fe4-df28bf72f4f0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:13.690744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:13.691362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:13.691635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:13.691901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] PATH_INFO: `/v3/users/b0a0fbd606f34d589c744a8949c7f12f` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:13.693005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a0fbd606f34d589c744a8949c7f12f)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:13.695346 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:13.741962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a0fbd606f34d589c744a8949c7f12f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:13.742763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a0fbd606f34d589c744a8949c7f12f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:13.750523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a0fbd606f34d589c744a8949c7f12f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:13.757838 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a0fbd606f34d589c744a8949c7f12f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:13.758460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a0fbd606f34d589c744a8949c7f12f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:13.776089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4239/8548] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:30:13 2022] DELETE /identity/v3/users/b0a0fbd606f34d589c744a8949c7f12f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:13.780822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:13.781301 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:13.781400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:13.800865 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:13.802072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:13.834731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:13.862020 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4310/8549] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:13.871968 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:13.873449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:13.887577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:13.938417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916e96d-52b4-46cd-8bb2-e30a778c41fd admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:13.942033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:13.942667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:13.942951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:13.943730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:13.952829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:13.954275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7cf9a-6c43-4eed-8502-4ca4f5429f0b service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:13.964760 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4240/8550] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:14.231163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.232653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:14.244897 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc74bf9-4d05-494b-af45-894bd763ffa4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.250060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.250740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.251024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.251300 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] PATH_INFO: `/v3/projects/a25a574ff0384f89b15634566b0a82b6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.252453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25a574ff0384f89b15634566b0a82b6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.254459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.307207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25a574ff0384f89b15634566b0a82b6', 'request_id': 'req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:14.313046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25a574ff0384f89b15634566b0a82b6', 'request_id': 'req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:14.320129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25a574ff0384f89b15634566b0a82b6', 'request_id': 'req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:14.332780 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] The token cache is being invalidate because project a25a574ff0384f89b15634566b0a82b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:30:14.333071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:14.333699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-a2fa3601-f7d6-407a-aa57-eb7f72d7fa0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:30:14.335507 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4311/8551] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:30:14 2022] DELETE /identity/v3/projects/a25a574ff0384f89b15634566b0a82b6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:14.340850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.341156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.341435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.361039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:14.362242 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:14.421022 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4241/8552] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:14.429331 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.429617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.429898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.451224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:14.452431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:14.460583 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4312/8553] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:14.464535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.474366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823faf8-c5bb-4d0d-8144-e19808901c12 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.479235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.479818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.480101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.480367 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.481024 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:30:14.481319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.482939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.494685 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4242/8554] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:30:14.498827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.508834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4f9d-3f2d-4423-a291-1807a853dd19 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.513627 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.514217 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.514518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.514794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.515451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.517337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.543814 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4313/8555] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:14.548052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.558872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f6ecb-2c0d-480a-8ba1-3a72471c13af admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.613907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.614531 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.614814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.615586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.616306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.618220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.641973 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4243/8556] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:14.644260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.655421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79793f1-7884-44ee-ab92-7fb39367e9e7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.660054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.660647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.660929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.661203 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.661837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.664107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.675465 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4314/8557] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:14.680818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.690854 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e35d6-5a84-4132-9cd4-853330c6678a admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.695795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.696382 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.696656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.696925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.697574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.699203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddcbe1-c6f5-42f9-8b73-f85243009383 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.709923 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4244/8558] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:14.713888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.723958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c8e4-d82b-49d1-a221-ba3c97bbebde admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.729263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.729876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.730157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.730434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] PATH_INFO: `/v3/projects/a99294b9428b4abc991cc9957afe73a0/users/fefd5257355844e590c72435f4c45024/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.746032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99294b9428b4abc991cc9957afe73a0, user_id=fefd5257355844e590c72435f4c45024, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.748734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.761228 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4315/8559] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:14 2022] PUT /identity/v3/projects/a99294b9428b4abc991cc9957afe73a0/users/fefd5257355844e590c72435f4c45024/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:14.773199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.773482 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.773757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.801658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] MFA Rules not processed for user `fefd5257355844e590c72435f4c45024`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:14.804107 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:14.812331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:14.869839 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4245/8560] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:14.874314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.936601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b1fe1-6e20-4376-be7d-5f5fbc55e471 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:14.941345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:14.941944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:14.942228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:14.942528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:14.943194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:14.945278 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:14.949712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:14.973058 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4316/8561] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:30:14.977900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:14.989028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db5276-5bd0-40f2-af98-78da8d58d156 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.045326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.045922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.046329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.046641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.047255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.049170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.077380 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4246/8562] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:14 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:15.078639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.089783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb584561-7bcd-423a-a5c0-66294887e7aa admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.094374 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.094994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.095270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.095541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.096166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.097748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.108285 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4317/8563] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:15.111954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.122600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a788dbe-e265-4246-8794-09fc9d769ff7 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.127428 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.128016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.128289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.128561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.129260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.131026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986abf6-c00d-440c-88c8-61d6005845a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.141004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4247/8564] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:15.144941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.156114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad04d-b9d1-4625-9ff5-94a0c67cd7f2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.160822 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.161464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.162591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.163029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] PATH_INFO: `/v3/projects/9a56290696be4a91afc080ae98bb52b0/users/1cac4ba563b745b58add85d40a595c35/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.177298 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a56290696be4a91afc080ae98bb52b0, user_id=1cac4ba563b745b58add85d40a595c35, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.180013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.192527 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4318/8565] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:15 2022] PUT /identity/v3/projects/9a56290696be4a91afc080ae98bb52b0/users/1cac4ba563b745b58add85d40a595c35/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:30:15.202826 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.203111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.203385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.232040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] MFA Rules not processed for user `1cac4ba563b745b58add85d40a595c35`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:15.233226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:15.298050 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4248/8566] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:30:15.302350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.364350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7408cc1-255b-4dd4-b65d-29627c5e53d7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.369208 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.369803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.370081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.370353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.370999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.372908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.399681 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4319/8567] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:30:15 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:15.403920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.415518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994970a8-54ad-45b4-a914-b209eff950a2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.427951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:15.473453 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.474060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.474342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.474709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.475319 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.477260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.498119 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4249/8568] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:30:15 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:30:15.502063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.513181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a38c8-5465-433a-97a3-1bd6bd80b517 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.517761 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.518345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.518663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.518934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.519554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.521128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.532762 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4320/8569] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:15.536855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.546779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bde21-1594-4998-add0-08086f789687 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.551419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.551998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.552274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.552559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] PATH_INFO: `/v3/projects/6022c96545bc424f834971fa50b630c8/users/8ef7462df2d94f46b76f0fb2c2d86d23/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.569709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6022c96545bc424f834971fa50b630c8, user_id=8ef7462df2d94f46b76f0fb2c2d86d23, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.572504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.585982 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4250/8570] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:15 2022] PUT /identity/v3/projects/6022c96545bc424f834971fa50b630c8/users/8ef7462df2d94f46b76f0fb2c2d86d23/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:15.589933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.600421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e18d44-5dad-4674-826b-e906cad32e8d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.657817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.658425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.658809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.659087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.659719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.661320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfe44-db47-49f8-bbcc-3794a93d93e5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.671743 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4321/8571] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:30:15.675931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.687721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e9fa9-f016-4bfc-9e9c-186e9e321812 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.692533 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.693185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.693580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.693855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] PATH_INFO: `/v3/projects/6022c96545bc424f834971fa50b630c8/users/8ef7462df2d94f46b76f0fb2c2d86d23/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.695779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6022c96545bc424f834971fa50b630c8, user_id=8ef7462df2d94f46b76f0fb2c2d86d23, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:15.698402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:15.708429 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4251/8572] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:30:15 2022] PUT /identity/v3/projects/6022c96545bc424f834971fa50b630c8/users/8ef7462df2d94f46b76f0fb2c2d86d23/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:30:15.718630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.718920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.719194 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:15.749642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] MFA Rules not processed for user `8ef7462df2d94f46b76f0fb2c2d86d23`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:15.750879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:15.758852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:15.819658 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4322/8573] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:30:15.845824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:15.852270 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:15.956887 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d8da1-f097-47b4-a6aa-eb56a86ce211 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:15.963442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:15.964028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:15.964301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:15.964569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:16.005137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:16.024464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:16.026063 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05719d82-68fc-4a12-8d7a-9ca30412b6f8 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:16.036503 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4252/8574] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:18.233752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:18.235229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:18.297299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261f16b-b0aa-4167-ad06-e2fb5b61db2e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:18.300958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:18.301565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:18.301845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:18.302120 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:18.312508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:18.313937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:18.324546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4323/8575] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:21.875223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:21.875696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:21.875782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:21.896490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] MFA Rules not processed for user `fefd5257355844e590c72435f4c45024`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:30:21.898352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:21.907457 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4253/8576] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:30:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:30:21.977998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:21.979578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:22.083197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1302a55a-61a7-43bf-8426-bf157319d1b4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:22.088288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:22.088919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:22.089199 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:22.089472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:22.091430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:22.111453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:22.112920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:22.129004 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4324/8577] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:22.555479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:22.568081 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb5b33-662a-494a-9507-92c2424ebba7 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:22.572150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:22.572749 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:22.573519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:22.573519 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:22.575058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:22.589452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:22.591216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:22.603772 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4254/8578] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:23.011157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:23.026353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6577c-438c-4901-a6b3-ad08a6d69b0a service glance] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:23.030895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:23.031576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:23.031924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:23.032261 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:23.034145 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:23.051661 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:23.053285 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:23.069238 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4325/8579] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:25.685144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:25.687857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:25.709124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd964e7-0bf3-4216-89a4-a54d88c101f3 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:25.715472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:25.716168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:25.716450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:25.716721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:25.718727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:25.734572 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:25.736189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1af488-e322-4350-907d-daac385bff6c service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:25.750531 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4255/8580] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:33.064543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:33.180462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:30:33.196317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87662934-2873-40a1-a5aa-0b4d8ea3513f service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:30:33.201968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:30:33.202606 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:30:33.202887 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:30:33.203172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:30:33.204901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:30:33.220736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:30:33.222307 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:30:33.236611 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4326/8581] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:30:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:30:35.743366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=24 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:30:43.229433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:31:48.768015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:31:48.768713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:31:48.768713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:31:48.795665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] MFA Rules not processed for user `fefd5257355844e590c72435f4c45024`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:31:48.798413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:31:48.811054 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4256/8582] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:31:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:31:49.158081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:31:49.173347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae245c5-c3a3-4e92-934c-76eb4476abb1 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:31:49.180706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:31:49.181330 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:31:49.181615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:31:49.181883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:31:49.183885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:31:49.208843 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:31:49.210532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:31:49.225902 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4327/8583] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:31:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:31:51.494570 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:31:51.508039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e648373-57c5-402a-a331-0ad6949661a8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:31:51.511748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:31:51.512606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:31:51.513053 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:31:51.513494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:31:51.516170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:31:51.530097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:31:51.532481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:31:51.546316 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4257/8584] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:31:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:31:59.218598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:31:59.377209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:31:59.389336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599cde2-6291-4fd4-b908-813c4f140ce1 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:31:59.393317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:31:59.393921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:31:59.394319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:31:59.394764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:31:59.396487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:31:59.409604 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:31:59.411102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:31:59.424423 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4328/8585] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:31:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:01.538751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:08.404929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:08.426632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e85f5-b8ca-462a-98a5-0da76fc2ef3e service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:08.433254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:08.433979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:08.434353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:08.434876 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:08.436830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:08.457085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:08.459070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afa90a-9538-43b8-bf16-a09fa5fe3ee3 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:08.475246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4258/8586] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:09.415790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:15.581956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:15.654584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca92e7-8a08-4b46-a098-cabbf4e0f4ac service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:15.660398 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:15.660950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:15.661225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:15.661492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] PATH_INFO: `/v3/users/fefd5257355844e590c72435f4c45024` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:15.662645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefd5257355844e590c72435f4c45024)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:15.664549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:15.715510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefd5257355844e590c72435f4c45024'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:15.716205 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefd5257355844e590c72435f4c45024'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:15.728016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefd5257355844e590c72435f4c45024'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:15.741405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefd5257355844e590c72435f4c45024'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:15.742155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefd5257355844e590c72435f4c45024'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:15.755139 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4329/8587] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:32:15 2022] DELETE /identity/v3/users/fefd5257355844e590c72435f4c45024 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:15.760879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:15.761338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:15.761437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:15.781656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:15.783609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:15.841417 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4259/8588] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:15.852657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:15.854072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:15.914609 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad31a4-5d18-4d75-ad86-86a11376eff3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:15.918281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:15.918917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:15.919197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:15.919470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:15.928745 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:15.930323 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:15.940682 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4330/8589] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:16.109821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:16.111512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:16.125160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404a1f6-8c74-4053-bec0-efe3a7f7bca3 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:16.130462 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:16.131131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:16.131937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:16.132217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] PATH_INFO: `/v3/projects/a99294b9428b4abc991cc9957afe73a0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:16.133540 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99294b9428b4abc991cc9957afe73a0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:16.135523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:16.183097 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99294b9428b4abc991cc9957afe73a0', 'request_id': 'req-b6566bfa-167c-4f99-b484-7dac7a848e8e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.189067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99294b9428b4abc991cc9957afe73a0', 'request_id': 'req-b6566bfa-167c-4f99-b484-7dac7a848e8e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.196383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99294b9428b4abc991cc9957afe73a0', 'request_id': 'req-b6566bfa-167c-4f99-b484-7dac7a848e8e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.208221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] The token cache is being invalidate because project a99294b9428b4abc991cc9957afe73a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:32:16.208508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.209135 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.210632 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4260/8590] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:32:16 2022] DELETE /identity/v3/projects/a99294b9428b4abc991cc9957afe73a0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:16.214544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:16.224917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a65d6-78a8-4b0b-87c0-145f30c1ac4a service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:16.242406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:16.283502 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:16.284381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:16.284657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:16.284926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] PATH_INFO: `/v3/users/1cac4ba563b745b58add85d40a595c35` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:16.286086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cac4ba563b745b58add85d40a595c35)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:16.288030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:16.334692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cac4ba563b745b58add85d40a595c35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.335439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cac4ba563b745b58add85d40a595c35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.344276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cac4ba563b745b58add85d40a595c35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.356148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cac4ba563b745b58add85d40a595c35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.356778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cac4ba563b745b58add85d40a595c35'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.369489 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4331/8591] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:32:16 2022] DELETE /identity/v3/users/1cac4ba563b745b58add85d40a595c35 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:16.651819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:16.653323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:16.667657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6566bfa-167c-4f99-b484-7dac7a848e8e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:16.726030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:16.726695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:16.726972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:16.727239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] PATH_INFO: `/v3/projects/9a56290696be4a91afc080ae98bb52b0` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:16.728338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a56290696be4a91afc080ae98bb52b0)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:16.730278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:16.742225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:16.780667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a56290696be4a91afc080ae98bb52b0', 'request_id': 'req-c19043e4-48b7-4e71-9823-5e61f96a290d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.786626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a56290696be4a91afc080ae98bb52b0', 'request_id': 'req-c19043e4-48b7-4e71-9823-5e61f96a290d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.793885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a56290696be4a91afc080ae98bb52b0', 'request_id': 'req-c19043e4-48b7-4e71-9823-5e61f96a290d'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.804977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] The token cache is being invalidate because project 9a56290696be4a91afc080ae98bb52b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:32:16.805265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.805929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.807431 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4261/8592] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:32:16 2022] DELETE /identity/v3/projects/9a56290696be4a91afc080ae98bb52b0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:16.811495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:16.812923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:16.827846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d282e7-18c1-4b7c-9d39-3b9cea46c11d admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:16.886224 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:16.886994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:16.887270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:16.887539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] PATH_INFO: `/v3/users/8ef7462df2d94f46b76f0fb2c2d86d23` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:16.888680 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef7462df2d94f46b76f0fb2c2d86d23)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:16.890873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:16.935518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:16.938891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef7462df2d94f46b76f0fb2c2d86d23'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.940043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef7462df2d94f46b76f0fb2c2d86d23'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.947860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef7462df2d94f46b76f0fb2c2d86d23'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.960429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef7462df2d94f46b76f0fb2c2d86d23'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.961079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-6976cefc-d626-4745-b536-4ccc925df031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef7462df2d94f46b76f0fb2c2d86d23'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:16.973276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4332/8593] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:32:16 2022] DELETE /identity/v3/users/8ef7462df2d94f46b76f0fb2c2d86d23 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:17.147796 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.149370 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:17.163607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19043e4-48b7-4e71-9823-5e61f96a290d admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.221718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.222348 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.222660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.225253 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] PATH_INFO: `/v3/projects/6022c96545bc424f834971fa50b630c8` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.229680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6022c96545bc424f834971fa50b630c8)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.231625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.278794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6022c96545bc424f834971fa50b630c8', 'request_id': 'req-48b915ac-6709-486f-bebd-03124e39c719'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:17.284695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6022c96545bc424f834971fa50b630c8', 'request_id': 'req-48b915ac-6709-486f-bebd-03124e39c719'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:17.292305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6022c96545bc424f834971fa50b630c8', 'request_id': 'req-48b915ac-6709-486f-bebd-03124e39c719'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:17.304675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] The token cache is being invalidate because project 6022c96545bc424f834971fa50b630c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:32:17.304959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:17.305582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-48b915ac-6709-486f-bebd-03124e39c719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:32:17.307168 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4262/8594] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:32:17 2022] DELETE /identity/v3/projects/6022c96545bc424f834971fa50b630c8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:17.312288 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.312580 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.312856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.333254 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:17.334461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:17.393910 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4333/8595] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:17.402131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.402413 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.402718 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.422423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:17.424504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:17.432347 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4263/8596] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:17.436686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.447430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beeca5-1c92-4fdc-b870-d11c716c1bd1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.451983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.452686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.452969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.453238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.453877 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:32:17.454418 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.456060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.467260 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4334/8597] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:32:17.471656 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.481005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea05ae-75ab-4d1b-b1b2-32d34a228d99 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.486330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.486954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.487230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.487526 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.488148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.490047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.518244 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4264/8598] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:32:17 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:32:17.522310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.533113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649237d8-c130-432d-b107-3dcfe1aed0f6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.589685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.590210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.590518 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.590798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.591419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.593325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.618205 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4335/8599] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:32:17 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:32:17.619508 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.625035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:17.631303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5cbe-1431-4106-8710-6b16eb304ae6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.636205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77679830-af07-4638-981b-088484a590dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.636833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77679830-af07-4638-981b-088484a590dc admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.637109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77679830-af07-4638-981b-088484a590dc admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.637379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77679830-af07-4638-981b-088484a590dc admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.638010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77679830-af07-4638-981b-088484a590dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.639764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77679830-af07-4638-981b-088484a590dc admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.651175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4265/8600] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:32:17.655216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.666168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9138bc-cd3e-43d1-a42e-2d5bd4c4e831 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.670874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.671464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.671739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.672009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.673303 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.674919 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be54af-e204-4246-808c-0055806897a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.684967 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4336/8601] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:32:17.689748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77679830-af07-4638-981b-088484a590dc admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.700202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77679830-af07-4638-981b-088484a590dc admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.704861 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.705501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.705799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.706058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] PATH_INFO: `/v3/projects/b690f172fad842e1a0f8d70a5ef27f5b/users/e8e099c4b8e74ce390c1893e70bcf600/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.720955 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b690f172fad842e1a0f8d70a5ef27f5b, user_id=e8e099c4b8e74ce390c1893e70bcf600, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.723668 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.737179 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4266/8602] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:32:17 2022] PUT /identity/v3/projects/b690f172fad842e1a0f8d70a5ef27f5b/users/e8e099c4b8e74ce390c1893e70bcf600/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:32:17.742175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b5e0-9860-4afc-82f3-9d17ec3d7b1a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.742460 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b5e0-9860-4afc-82f3-9d17ec3d7b1a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.742764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1b5e0-9860-4afc-82f3-9d17ec3d7b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.762845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-52e1b5e0-9860-4afc-82f3-9d17ec3d7b1a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:17.764032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-52e1b5e0-9860-4afc-82f3-9d17ec3d7b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:17.784470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:17.825128 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4337/8603] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:17.834947 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:17.836798 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:17.896085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19174fa-f035-49a4-8b8f-e6353d490347 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:17.899739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:17.900330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:17.900606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:17.900875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:17.910710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:17.912280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:17.921674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4267/8604] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:22.093993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.095151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.095151 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.128251 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] MFA Rules not processed for user `e8e099c4b8e74ce390c1893e70bcf600`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:22.129388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:22.198163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4338/8605] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:22.202862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.213142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698dfa-a87c-420b-b017-946a739629de service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.220007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.220990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.221338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.221675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.222578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.224654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.252335 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4268/8606] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:32:22 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:32:22.256538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.267996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418bfe1-8098-4841-9590-926e2acf6416 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.332117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.332731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.333006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.333272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.333909 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.335935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.359115 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4339/8607] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:32:22 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:32:22.364213 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.374875 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed314734-e044-4fde-afd9-acc99c2fbb48 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.379896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.380494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.380811 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.381125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.381915 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.383632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.397389 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4269/8608] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:32:22.400346 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.411720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc09bf-a21b-4600-a5f2-479d3363b6d1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.420086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.421481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.421481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.421481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] PATH_INFO: `/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6/users/42de83bfe0634ecb8480bee4d37c37ea/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.452709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c5742657f0487f975e5bfdbd5fc0f6, user_id=42de83bfe0634ecb8480bee4d37c37ea, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.456040 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.470480 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4340/8609] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:32:22 2022] PUT /identity/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6/users/42de83bfe0634ecb8480bee4d37c37ea/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:32:22.474517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.494061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc9748-6466-429c-bef8-3031ad1bdb19 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.557293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.557907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.558255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.558437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.559117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.561045 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138fa1-3914-4e84-9f51-4c537350d8e2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.574686 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4270/8610] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:32:22.576972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:22.588592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c79fa-cb7a-4182-8104-45ff5ef846a2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:22.593458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:22.594132 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:22.594406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:22.595387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] PATH_INFO: `/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6/users/42de83bfe0634ecb8480bee4d37c37ea/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:22.597599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3c5742657f0487f975e5bfdbd5fc0f6, user_id=42de83bfe0634ecb8480bee4d37c37ea, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:22.600615 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:22.618341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4341/8611] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:32:22 2022] PUT /identity/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6/users/42de83bfe0634ecb8480bee4d37c37ea/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:32:27.184694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442e204-9685-4f22-90cd-e4175104e2ce None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:27.185257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442e204-9685-4f22-90cd-e4175104e2ce None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:27.185257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442e204-9685-4f22-90cd-e4175104e2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:27.213568 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3442e204-9685-4f22-90cd-e4175104e2ce None None] MFA Rules not processed for user `42de83bfe0634ecb8480bee4d37c37ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:27.214824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3442e204-9685-4f22-90cd-e4175104e2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:27.237403 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:27.291132 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4271/8612] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:27.307163 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:27.309573 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:27.322890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:27.394634 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6824c-a439-4317-a1c9-2fe81e333f34 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:27.401742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:27.402365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:27.402687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:27.402966 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:27.462116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:27.463746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:27.473890 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4342/8613] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:28.706917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:28.707404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:28.707491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:28.727038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] MFA Rules not processed for user `e8e099c4b8e74ce390c1893e70bcf600`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:32:28.728156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:28.734920 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4272/8614] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:32:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:32:29.437684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:29.439549 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:29.523098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bf0c9-b81e-417c-b651-2c7fcd9454a3 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:29.528403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:29.529104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:29.529451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:29.529792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:29.531608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:32:29.547506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:29.549690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:29.563730 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4343/8615] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:32:38.721517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=22 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:39.555927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:32:40.694863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:32:40.753522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a55d76-9fc5-4705-aeff-c09c3d2216af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:32:40.757470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:32:40.758068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:32:40.758340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:32:40.758645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:32:40.768209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:32:40.769675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7494273-abb2-4f0d-89c9-0f41fc5dbedd service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:32:40.778975 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4273/8616] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:32:50.774830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=26 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:33:10.659925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:33:10.674360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7255b-7301-46e4-a25e-69c8065f05c5 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:33:10.679802 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:33:10.680390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:33:10.680665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:33:10.680930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:33:10.682706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:33:10.699019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:33:10.700675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a150-f221-4e4d-bb28-a305cbb58820 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:33:10.714093 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4344/8617] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:33:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:33:17.690958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:33:17.691541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:33:17.691541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:33:17.692967 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4274/8618] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:33:17.698776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:33:17.699185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:33:17.699318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:33:17.746775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:33:17.747965 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:33:17.754353 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4345/8619] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:33:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:33:17.766846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:33:17.768488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:33:17.784739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486deac-44a4-4337-b88b-af98ab769bd6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:33:17.789155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:33:17.789739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:33:17.790013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:33:17.790278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:33:17.797188 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:33:17.798631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea472f0-1e1e-44a8-848d-37896617fc2f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:33:17.806270 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4275/8620] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:33:20.358992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:33:20.374392 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17416-9571-4d3f-aa90-cd755048668e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:33:20.379484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:33:20.380113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:33:20.380393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:33:20.380668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:33:20.382974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:33:20.397988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:33:20.399605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:33:20.412373 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4346/8621] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:33:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:33:27.802910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:33:30.405358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=25 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:19.057404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:19.057981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:19.057981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:19.108017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:19.109220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:19.135764 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4276/8622] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 21:37:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:19.141633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:19.160470 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef7dae-d6b7-43fa-ad6a-f3216460ccc1 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:19.164753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:19.165372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:19.165653 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:19.165927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:19.167749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:19.180997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:19.182416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:19.192815 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4347/8623] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:37:22.233416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:22.242173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426b310-4aee-434b-b5af-303ebcaa73c0 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:22.245857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:22.246448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:22.246758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:22.247031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:22.248744 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:22.285156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:22.286637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:22.300854 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4277/8624] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:23.497617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:23.506117 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc95b1b-969f-4c46-a6d2-d0477f06af0e service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:23.510202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:23.510841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:23.511102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:23.511371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:23.513119 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:23.530192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:23.531650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3f502-a657-4294-9071-9e80582d5452 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:23.546770 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4348/8625] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:24.965452 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:24.985869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730f0-d799-4292-9566-c2e0b8492b60 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:24.996132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:24.996945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:24.997229 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:24.997506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:24.999576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:25.020092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:25.021795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:25.037402 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4278/8626] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:26.608246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:26.608880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:26.608880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:26.630382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:26.631671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:26.642859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4349/8627] 10.222.0.52 () {58 vars in 1061 bytes} [Sat Sep 24 21:37:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:26.649059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:26.658654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f64e4-7626-4020-a821-07ea28e35011 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:26.662307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:26.662933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:26.663217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:26.663528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:26.733277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:26.734759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:26.744896 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4279/8628] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:37:29.885412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:29.902214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99040111-9d3e-4461-add4-144bf2682e41 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:29.907484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:29.908106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:29.908386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:29.908659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:29.910734 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:29.926196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:29.927869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:29.942883 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4350/8629] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:33.135776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:33.156750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c4df1-9e08-4434-b901-d0817a6b1634 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:33.164195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:33.165218 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:33.165661 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:33.166080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:33.168189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:33.190789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:33.202953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:33.215550 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:33.219562 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4280/8630] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:33.238155 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1d05a-f2e3-4829-8253-09159ba43a17 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:33.248107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:33.248842 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:33.249329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:33.249678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] PATH_INFO: `/v3/users/e8e099c4b8e74ce390c1893e70bcf600` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:33.250971 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e099c4b8e74ce390c1893e70bcf600)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:33.253215 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:33.306060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e099c4b8e74ce390c1893e70bcf600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.306784 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e099c4b8e74ce390c1893e70bcf600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.314225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e099c4b8e74ce390c1893e70bcf600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.327993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e099c4b8e74ce390c1893e70bcf600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.328689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e099c4b8e74ce390c1893e70bcf600'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.341150 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4351/8631] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:37:33 2022] DELETE /identity/v3/users/e8e099c4b8e74ce390c1893e70bcf600 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:37:33.520380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:33.531296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b42ec1-2d01-496a-9ea2-3ef8cc1b9688 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:33.569910 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:33.597472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:33.598088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:33.598362 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:33.598664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] PATH_INFO: `/v3/projects/b690f172fad842e1a0f8d70a5ef27f5b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:33.599763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b690f172fad842e1a0f8d70a5ef27f5b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:33.602191 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:33.649042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b690f172fad842e1a0f8d70a5ef27f5b', 'request_id': 'req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.654645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b690f172fad842e1a0f8d70a5ef27f5b', 'request_id': 'req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.662405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b690f172fad842e1a0f8d70a5ef27f5b', 'request_id': 'req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.687096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] The token cache is being invalidate because project b690f172fad842e1a0f8d70a5ef27f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:37:33.687306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.687930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.689408 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4281/8632] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:37:33 2022] DELETE /identity/v3/projects/b690f172fad842e1a0f8d70a5ef27f5b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:37:33.693438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:33.694962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:33.708161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea96056-6e81-4aa9-b32a-163ac4738572 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:33.764520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:33.765147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:33.765424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:33.765697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] PATH_INFO: `/v3/users/42de83bfe0634ecb8480bee4d37c37ea` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:33.766860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=42de83bfe0634ecb8480bee4d37c37ea)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:33.768811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:33.784416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:33.816926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42de83bfe0634ecb8480bee4d37c37ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.817570 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42de83bfe0634ecb8480bee4d37c37ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.824958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42de83bfe0634ecb8480bee4d37c37ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.836980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42de83bfe0634ecb8480bee4d37c37ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.837608 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-83722ee7-b8cb-4b4c-8c19-3ac496b94bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42de83bfe0634ecb8480bee4d37c37ea'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:33.849882 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4352/8633] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:37:33 2022] DELETE /identity/v3/users/42de83bfe0634ecb8480bee4d37c37ea => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:37:34.027332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:34.028872 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:34.042957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81cb84-1f16-4d2b-8afd-dd782f6b53be admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:34.106408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:34.107068 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:34.107351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:34.107626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] PATH_INFO: `/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:34.108776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3c5742657f0487f975e5bfdbd5fc0f6)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:34.110734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:34.157113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3c5742657f0487f975e5bfdbd5fc0f6', 'request_id': 'req-17583c75-ea10-4693-aee6-fcb042e6a14f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:34.159060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:34.164236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3c5742657f0487f975e5bfdbd5fc0f6', 'request_id': 'req-17583c75-ea10-4693-aee6-fcb042e6a14f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:34.172015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3c5742657f0487f975e5bfdbd5fc0f6', 'request_id': 'req-17583c75-ea10-4693-aee6-fcb042e6a14f'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:34.183564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] The token cache is being invalidate because project e3c5742657f0487f975e5bfdbd5fc0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:37:34.183857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:34.184478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-17583c75-ea10-4693-aee6-fcb042e6a14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:37:34.185965 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4282/8634] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:37:34 2022] DELETE /identity/v3/projects/e3c5742657f0487f975e5bfdbd5fc0f6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:37:43.846370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:44.180671 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6,keystone:DELETE=2,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:44.655365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:44.655646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:44.655913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:44.703326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:44.704526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:44.763449 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4353/8635] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:44.772541 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:44.772834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:44.773110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:44.793246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:44.794446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:44.802464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4283/8636] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:44.807185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:44.816950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd87e0-4be8-4291-9554-49fef7486d57 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:44.822051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:44.822805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:44.823081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:44.823351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:44.823987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:37:44.824287 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:44.825886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:44.837247 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4354/8637] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:37:44.846140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:44.855566 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ef9da-afe7-4ae8-b0a6-987b89de38cf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:44.860294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:44.860881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:44.861156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:44.861425 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:44.862041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:44.863988 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:44.900840 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4284/8638] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:37:44 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:37:44.905136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:44.915105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37ee42-7842-479c-bed1-3ea99d249c8e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:44.972600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:44.973305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:44.973591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:44.973905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:44.974547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:44.976475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:44.997393 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4355/8639] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:37:44 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:37:45.001546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.012563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e19269b-2f03-4953-8f13-7ec72247d25c admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.017244 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.017842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.018151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.018426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.019088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.020722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.031096 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4285/8640] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:37:45.035029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.045033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521aff-6682-472e-b076-7652a1ca6985 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.049953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.050567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.050879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.051130 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.051754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.053335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d6162-cd3b-4afe-a4eb-58169dd9ce85 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.070197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4356/8641] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:37:45.074970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.085172 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f892e64-8485-4845-adf4-1c704112dd4b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.090787 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.091435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.091714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.091986 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] PATH_INFO: `/v3/projects/81e0a49f48ed4f399bc339fbf37e75de/users/712476b96d7c4066a460868988d23109/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.108145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e0a49f48ed4f399bc339fbf37e75de, user_id=712476b96d7c4066a460868988d23109, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.110845 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.124127 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4286/8642] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:37:45 2022] PUT /identity/v3/projects/81e0a49f48ed4f399bc339fbf37e75de/users/712476b96d7c4066a460868988d23109/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:37:45.138121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.138405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.138707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.166133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] MFA Rules not processed for user `712476b96d7c4066a460868988d23109`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:45.167304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:45.200313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:45.234168 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4357/8643] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:45.238731 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.305271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5b22-2f8a-4d8d-9590-a391c028f0a4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.310778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.311368 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.311642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.311916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.312522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.314417 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.343425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4287/8644] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:37:45 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:37:45.347347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.358181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c48a49-d33c-49ff-8a2b-a16d1a3423be None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.414030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.414671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.414953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.415247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.416189 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.418325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.442533 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4358/8645] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:37:45 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:37:45.443898 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.455675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105180c4-f319-4358-8811-6c534bd97d9e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.460287 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.460900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.461175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.461440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.462061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.464090 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.474327 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4288/8646] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:37:45.479200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.489361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a5b3-55cd-4554-99f9-3d984c76257a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.494614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.495258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.495539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.495815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] PATH_INFO: `/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d/users/1fa85cce135249b1acfa5a0d22d7bdbc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.512341 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abb60b8f1a4468e86a4b37b33dd287d, user_id=1fa85cce135249b1acfa5a0d22d7bdbc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.515229 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.528455 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4359/8647] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:37:45 2022] PUT /identity/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d/users/1fa85cce135249b1acfa5a0d22d7bdbc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:37:45.532422 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.542992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f140d-b45f-4263-ac70-badb506c6f51 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.598933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.599529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.599809 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.600079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.600706 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.602300 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225104f9-afb9-4943-ba9e-db4ef23307d2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.604823 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:45.614969 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4289/8648] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:37:45.619687 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.629998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f97ce4-eabc-412d-82a3-2a8247472be7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.635668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.635668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.636442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.636442 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] PATH_INFO: `/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d/users/1fa85cce135249b1acfa5a0d22d7bdbc/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.637987 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abb60b8f1a4468e86a4b37b33dd287d, user_id=1fa85cce135249b1acfa5a0d22d7bdbc, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.640785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a175-2f6a-42cf-9475-6f2f1d2b687b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.651631 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4360/8649] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:37:45 2022] PUT /identity/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d/users/1fa85cce135249b1acfa5a0d22d7bdbc/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:37:45.661772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.662062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.662338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.691575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] MFA Rules not processed for user `1fa85cce135249b1acfa5a0d22d7bdbc`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:45.692693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:45.761437 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4290/8650] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:45.774694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.774988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.775266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.794525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] MFA Rules not processed for user `712476b96d7c4066a460868988d23109`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:37:45.795732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:45.850079 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4361/8651] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:37:45.864544 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:45.866373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:45.942690 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd603ce-5cf1-400c-b30c-96c120672ccc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:45.948058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:45.948658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:45.948935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:45.949205 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:45.956998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:45.958616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:45.968142 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4291/8652] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:46.431525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:46.432929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:46.473492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:46.514112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83095b9a-b7a4-472a-bb61-f7816ea43f6e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:46.519281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:46.519885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:46.520166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:46.520439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:46.522152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:46.536726 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:46.538310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:46.550957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4362/8653] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:48.374919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:48.376434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:48.439437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a134a-844e-485f-95e7-25e36af8cc5b service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:48.443082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:48.443689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:48.443963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:48.444231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:48.445934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:48.461881 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:48.464268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:48.475908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4292/8654] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:55.510941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:55.529258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7c71d-1df3-4fdb-ae2c-e6e5f1f69024 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:55.534815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:55.535420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:55.535700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:55.535973 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:55.537736 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:55.553195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:55.554807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1c9f4-700b-4be5-a824-88a1108f6294 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:55.568526 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4363/8655] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:58.319442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:58.335310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a3ae3-8719-4eb3-8133-f16f54e3c8ae service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:58.341528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:58.342113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.342390 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.342691 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.351784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:58.353405 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:58.357331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:37:58.363739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4293/8656] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:58 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:58.378576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cbb93-04fd-4842-81be-4f6e77d983f5 None None] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.378999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cbb93-04fd-4842-81be-4f6e77d983f5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.379098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cbb93-04fd-4842-81be-4f6e77d983f5 None None] PATH_INFO: `/` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.380415 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4364/8657] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:37:58.386185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] Authenticating service token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:37:58.401011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:58.409266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50ce9-b500-4b26-86b7-dbd6787e3be2 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:58.413846 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fa85cce135249b1acfa5a0d22d7bdbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4abb60b8f1a4468e86a4b37b33dd287d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:58.414454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.414769 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.415080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] PATH_INFO: `/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.416305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: Authorizing `identity:get_project(project_id=4abb60b8f1a4468e86a4b37b33dd287d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:58.418075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d13b07-18cf-43a9-a40b-03d58da0584b tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:58.419711 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4294/8658] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:37:58 2022] GET /identity/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d => generated 393 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:37:58.576456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cbb93-04fd-4842-81be-4f6e77d983f5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:58.587166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cbb93-04fd-4842-81be-4f6e77d983f5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:58.591495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:58.592085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.592370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.592647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.594366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:58.606825 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:58.608253 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:58.618760 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4365/8659] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:37:58.787446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b87c-d94b-4f71-9841-23c03e61e968 None None] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.787733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b87c-d94b-4f71-9841-23c03e61e968 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.788149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94b87c-d94b-4f71-9841-23c03e61e968 None None] PATH_INFO: `/` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.789496 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4295/8660] 10.222.0.52 () {54 vars in 912 bytes} [Sat Sep 24 21:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 21:37:58.794436 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] Authenticating service token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} Sep 24 21:37:58.795861 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:58.815555 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:58.825633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d03b3c-173c-49b3-a64f-0fff8a71df5d service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:58.830223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fa85cce135249b1acfa5a0d22d7bdbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4abb60b8f1a4468e86a4b37b33dd287d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:58.830878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:58.831159 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:58.831431 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] PATH_INFO: `/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:58.832551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: Authorizing `identity:get_project(project_id=4abb60b8f1a4468e86a4b37b33dd287d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:58.834320 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:58.835970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4366/8661] 10.222.0.52 () {58 vars in 1443 bytes} [Sat Sep 24 21:37:58 2022] GET /identity/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d => generated 393 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:37:58.991129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b87c-d94b-4f71-9841-23c03e61e968 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:37:59.002827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94b87c-d94b-4f71-9841-23c03e61e968 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:37:59.006419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:37:59.007044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:37:59.007325 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:37:59.007600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:37:59.009336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:37:59.021439 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:37:59.022964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e083a0-d4c7-4726-b0eb-599e77ff54f7 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:37:59.033604 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4296/8662] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:37:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:03.103390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:03.171495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef4bc0-570b-45ab-a2f1-9060dba94651 tempest-ServersAdminNegativeTestJSON-1935027202 tempest-ServersAdminNegativeTestJSON-1935027202-project-admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:03.176098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:03.176667 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:03.176948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:03.177230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] PATH_INFO: `/v3/users/712476b96d7c4066a460868988d23109` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:03.178334 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=712476b96d7c4066a460868988d23109)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:03.180284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:03.226264 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712476b96d7c4066a460868988d23109'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.226952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712476b96d7c4066a460868988d23109'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.234066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712476b96d7c4066a460868988d23109'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.246456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712476b96d7c4066a460868988d23109'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.247133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712476b96d7c4066a460868988d23109'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.259011 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4367/8663] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:38:03 2022] DELETE /identity/v3/users/712476b96d7c4066a460868988d23109 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:38:03.264231 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:03.264695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:03.264800 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:03.283822 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:03.285012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:03.344662 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4297/8664] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:03.354166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:03.356024 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:03.362530 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:03.416625 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9846d4d-0bc0-4c9e-b9f8-fc4c1eec13e0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:03.420678 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:03.421536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:03.421838 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:03.422104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:03.431173 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:03.432528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:03.442957 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4368/8665] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:03.612479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:03.613899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:03.626906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b2d6a-07fc-4ac0-9216-69ff755851d6 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:03.632026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:03.632745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:03.633095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:03.633440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] PATH_INFO: `/v3/projects/81e0a49f48ed4f399bc339fbf37e75de` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:03.638550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e0a49f48ed4f399bc339fbf37e75de)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:03.638550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:03.692140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e0a49f48ed4f399bc339fbf37e75de', 'request_id': 'req-8b0268af-f9a5-4879-b84b-9cd18c311d11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.697692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e0a49f48ed4f399bc339fbf37e75de', 'request_id': 'req-8b0268af-f9a5-4879-b84b-9cd18c311d11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.705035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e0a49f48ed4f399bc339fbf37e75de', 'request_id': 'req-8b0268af-f9a5-4879-b84b-9cd18c311d11'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.717010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] The token cache is being invalidate because project 81e0a49f48ed4f399bc339fbf37e75de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:38:03.717298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.717938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.719490 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4298/8666] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:38:03 2022] DELETE /identity/v3/projects/81e0a49f48ed4f399bc339fbf37e75de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:38:03.723619 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:03.733963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62699d7a-ea85-4cf2-bc24-9972c35b63ca service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:03.788491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:03.789113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:03.789394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:03.789664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] PATH_INFO: `/v3/users/1fa85cce135249b1acfa5a0d22d7bdbc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:03.790812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa85cce135249b1acfa5a0d22d7bdbc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:03.793379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:03.838548 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa85cce135249b1acfa5a0d22d7bdbc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.839707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa85cce135249b1acfa5a0d22d7bdbc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.849180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa85cce135249b1acfa5a0d22d7bdbc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.862172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa85cce135249b1acfa5a0d22d7bdbc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.862858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-81965d08-4d2c-4e38-9307-ef4467571b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa85cce135249b1acfa5a0d22d7bdbc'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:03.875291 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4369/8667] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:38:03 2022] DELETE /identity/v3/users/1fa85cce135249b1acfa5a0d22d7bdbc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:38:04.150548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.152597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:04.168162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0268af-f9a5-4879-b84b-9cd18c311d11 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.226993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.227538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.227821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.228096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] PATH_INFO: `/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.229223 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abb60b8f1a4468e86a4b37b33dd287d)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.231341 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.232726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:04.281309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abb60b8f1a4468e86a4b37b33dd287d', 'request_id': 'req-8f44485a-9c84-42a0-b715-670630c0b373'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:04.286806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abb60b8f1a4468e86a4b37b33dd287d', 'request_id': 'req-8f44485a-9c84-42a0-b715-670630c0b373'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:04.294232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abb60b8f1a4468e86a4b37b33dd287d', 'request_id': 'req-8f44485a-9c84-42a0-b715-670630c0b373'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:04.305595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] The token cache is being invalidate because project 4abb60b8f1a4468e86a4b37b33dd287d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:38:04.305870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:04.306522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-8f44485a-9c84-42a0-b715-670630c0b373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:38:04.308071 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4299/8668] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:38:04 2022] DELETE /identity/v3/projects/4abb60b8f1a4468e86a4b37b33dd287d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:38:04.313994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.314461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.314589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.334266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:04.335504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:04.356259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:04.397922 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4370/8669] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:04.406704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.406993 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.407269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.427091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:04.428299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:04.436272 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4300/8670] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:04.440613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.451231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ab358-f02d-4efe-8e5c-c5033d92bfd5 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.455951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.456551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.456832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.457105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.457775 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:38:04.458081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.459766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.472111 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4371/8671] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:38:04.476347 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.495049 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d14ad-7ad2-469b-a309-88bc15e2bf9c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.499941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.500542 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.500821 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.501095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.501720 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.503651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.531501 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4301/8672] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:38:04 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:38:04.535656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.556177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88474-b33d-45a8-a7e3-aede7ca96284 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.613526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.614121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.614402 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.614701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.615328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.617241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.642222 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4372/8673] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:38:04 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:38:04.643509 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.653973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eebf81-2ed9-43b9-89c0-0fe97e7cf10b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.658583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.659167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.659441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.659705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.660328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.661963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.672317 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4302/8674] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:38:04.676289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.687400 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f653e-ec31-45f1-8487-4e9667a150f5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.691968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.692547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.692821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.693089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.693772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.695393 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84783728-5a72-4886-9f3b-29518a67e5cd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.706240 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4373/8675] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:38:04.710698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.720645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec66b2e-2a3f-4a28-940f-6ded1dc348ec admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.725299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.725964 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.726245 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.726547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] PATH_INFO: `/v3/projects/d1128ece39524084b4e4229ad0375bf4/users/0bff3127ccc748e49e7d6d5d34df43ad/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.740763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1128ece39524084b4e4229ad0375bf4, user_id=0bff3127ccc748e49e7d6d5d34df43ad, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.743457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.756715 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4303/8676] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:38:04 2022] PUT /identity/v3/projects/d1128ece39524084b4e4229ad0375bf4/users/0bff3127ccc748e49e7d6d5d34df43ad/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:38:04.762363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e72de-29f9-4ba5-9444-23acbaee0ae6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.762682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e72de-29f9-4ba5-9444-23acbaee0ae6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.762958 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e72de-29f9-4ba5-9444-23acbaee0ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.782275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-733e72de-29f9-4ba5-9444-23acbaee0ae6 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:04.783511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-733e72de-29f9-4ba5-9444-23acbaee0ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:04.841915 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4374/8677] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:04.851460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:04.853161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:04.919254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7b73c-b51a-41b1-85e1-d8e3ea35cb60 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:04.923401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:04.924005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:04.924282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:04.924550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:04.934278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:04.935771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bca1-b953-4914-86e6-37b881e07d40 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:04.945194 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4304/8678] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:08.896698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:08.897214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:08.897453 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:08.928633 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] MFA Rules not processed for user `0bff3127ccc748e49e7d6d5d34df43ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:08.930007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:08.947039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:09.021807 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4375/8679] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:09.033059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998b62-9216-4fdb-abe5-bfb783155e08 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:09.033593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998b62-9216-4fdb-abe5-bfb783155e08 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:09.033593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7998b62-9216-4fdb-abe5-bfb783155e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:09.055518 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d7998b62-9216-4fdb-abe5-bfb783155e08 None None] MFA Rules not processed for user `0bff3127ccc748e49e7d6d5d34df43ad`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:09.056512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d7998b62-9216-4fdb-abe5-bfb783155e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:09.095208 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4305/8680] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:38:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:09.099166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:09.100440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:09.196716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae5989-eca9-4bd2-839c-b62b2374dddc None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:09.203426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:09.204185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:09.204671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:09.205012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:09.214409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:09.216239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:09.231792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4376/8681] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:10.851966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:10.852399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:10.852507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:10.861388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:10.917864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:10.919309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:11.019187 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4306/8682] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:38:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:11.032685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:11.034195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:11.054395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5200c-5dc0-458c-bd6f-9c6c183113d6 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:11.068098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:11.068953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:11.069235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:11.069506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:11.079917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:11.080699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:11.093092 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4377/8683] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:13.683186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:13.684606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:13.710082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb95cc-74a0-4063-84b8-ff21c8e5050d None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:13.717016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:13.717778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:13.718066 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:13.718345 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:13.720180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:13.734434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:13.736032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:13.749070 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4307/8684] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:21.088181 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=61 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:23.327870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:23.351938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48477ecb-5607-4424-a8ee-bc688e67021a service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:23.357879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:23.358475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:23.358787 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:23.359065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:23.360807 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:23.374952 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:23.376511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57c64-79f2-4e7a-8ce6-f5a8769e861d service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:23.389339 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4378/8685] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:23.741711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=48 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:24.393198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:24.403767 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb951b-c3bb-4f79-b8a9-c35987019b71 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:24.407392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:24.408050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.408299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.408576 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.410358 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:24.424850 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:24.426289 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:24.437480 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4308/8686] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:24.548770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.549269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.549356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.570376 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:24.571655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:24.580922 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4379/8687] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:38:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:24.592931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:24.607938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0a423-2345-4c84-994a-992811f0bbc1 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:24.613174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:24.613761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.614044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.614316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.622978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:24.624547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcc913-0b56-48ba-adb9-faaf7873903d service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:24.635215 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4309/8688] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:24.820996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:24.830890 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22f371-99ac-463c-bf47-d057fe1f454e None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:24.834434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:24.835060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.835339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.835611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.837313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:24.853699 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:24.855175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:24.868640 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4380/8689] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:38:24.923352 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.923841 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.923945 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.944454 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:38:24.945680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:38:24.955109 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4310/8690] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:38:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:38:24.966484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:38:24.980967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629fc63-6882-42ef-8ab9-255bb1cf20d4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:38:24.986043 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:38:24.986670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:38:24.986954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:38:24.987226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:38:24.996113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:38:24.997676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:38:25.007441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4381/8691] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:38:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:38:34.939585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=15 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:38:35.003277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=25 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:39:04.970550 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:04.988419 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e32e0-7ec7-429a-99fd-9897ea92f0e5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:04.993557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.005851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.006163 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.006447 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] PATH_INFO: `/v3/users/0bff3127ccc748e49e7d6d5d34df43ad` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.007994 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bff3127ccc748e49e7d6d5d34df43ad)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.010162 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.067734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bff3127ccc748e49e7d6d5d34df43ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.068554 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bff3127ccc748e49e7d6d5d34df43ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.077472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bff3127ccc748e49e7d6d5d34df43ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.085449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bff3127ccc748e49e7d6d5d34df43ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.086371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-2b27d1c2-6b14-4c8e-966e-fe27afc3e97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bff3127ccc748e49e7d6d5d34df43ad'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.110681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4311/8692] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:39:04 2022] DELETE /identity/v3/users/0bff3127ccc748e49e7d6d5d34df43ad => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:39:05.288662 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.307095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01e006-c3d8-4f90-886a-1299492d80a3 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.370017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.370614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.370900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.371176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] PATH_INFO: `/v3/projects/d1128ece39524084b4e4229ad0375bf4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.372308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1128ece39524084b4e4229ad0375bf4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.374220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.421272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1128ece39524084b4e4229ad0375bf4', 'request_id': 'req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.427000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1128ece39524084b4e4229ad0375bf4', 'request_id': 'req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.433349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1128ece39524084b4e4229ad0375bf4', 'request_id': 'req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.446087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] The token cache is being invalidate because project d1128ece39524084b4e4229ad0375bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:39:05.446380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.447076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-9bff3f0d-fcd0-4e23-b722-a36b4ed205ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:39:05.448603 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4382/8693] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:39:05 2022] DELETE /identity/v3/projects/d1128ece39524084b4e4229ad0375bf4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:39:05.453829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.454179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.454455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.475394 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:39:05.477498 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:05.541949 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4312/8694] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:39:05.551704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.551991 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.552270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.571994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:39:05.573201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:05.581116 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4383/8695] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:39:05.585402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.595755 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5fdb2-d2c9-4cb9-ae2e-c8ea7f20d8c4 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.600523 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.601114 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.601388 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.601657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.602322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:39:05.602657 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.604238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.616628 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4313/8696] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:39:05.621277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.631279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d522-3c46-4760-b713-6d0182b4f3d8 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.636207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.636811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.637108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.637982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.638654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.640562 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.668715 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4384/8697] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:39:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:39:05.673312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.683179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f099-bba4-4213-848a-4df5d96b0801 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.740489 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.741091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.741373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.741643 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.742288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.744278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.752147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:39:05.767229 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4314/8698] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:39:05 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:39:05.771584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.782821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3a45-0943-4b3e-9352-8d67a65e56e1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.787611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.788232 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.788515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.788783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.789408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.791051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.801253 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4385/8699] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:39:05.805291 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.815410 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcfbb-c96e-402a-86c1-f20697745dbf admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.820117 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.820709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.820990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.821266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.821899 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.823562 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed914fb-5f7f-492d-bc25-1d3101ce41b3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.834721 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4315/8700] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:39:05.838806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.849594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd44ff-9b7e-4360-9739-e6f947d55ad4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:05.854643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:05.855792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.856305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.856645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] PATH_INFO: `/v3/projects/6a5a6205a2c74f07bda8d3bf7b5a5e6d/users/45c51ff8103f42fa9d68a5492904b315/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.870951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5a6205a2c74f07bda8d3bf7b5a5e6d, user_id=45c51ff8103f42fa9d68a5492904b315, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:05.873721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:05.886054 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4386/8701] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:39:05 2022] PUT /identity/v3/projects/6a5a6205a2c74f07bda8d3bf7b5a5e6d/users/45c51ff8103f42fa9d68a5492904b315/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:39:05.891761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1d78-e739-427d-92ae-852b0fa6cfea None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:05.892074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1d78-e739-427d-92ae-852b0fa6cfea None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:05.892349 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf1d78-e739-427d-92ae-852b0fa6cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:05.912252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cdcf1d78-e739-427d-92ae-852b0fa6cfea None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:39:05.913469 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cdcf1d78-e739-427d-92ae-852b0fa6cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:05.972271 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4316/8702] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:39:05.982804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:05.984521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:06.016351 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:39:06.045630 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc38ca-74a4-4682-9acf-c07e3a447708 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:06.049638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:06.050239 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:06.050542 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:06.050820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:06.059808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:06.061255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c5fe-02c6-4a18-8d4f-a0bd0ff37b76 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:06.070559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4387/8703] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:39:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:39:09.870130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:09.870692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:09.870692 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:09.902423 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] MFA Rules not processed for user `45c51ff8103f42fa9d68a5492904b315`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:39:09.904171 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:09.972364 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4317/8704] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:39:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:39:09.983139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b499-a14b-4dd3-b8b5-778e50bec454 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:09.983644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b499-a14b-4dd3-b8b5-778e50bec454 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:09.983644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8b499-a14b-4dd3-b8b5-778e50bec454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:10.004879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f9b8b499-a14b-4dd3-b8b5-778e50bec454 None None] MFA Rules not processed for user `45c51ff8103f42fa9d68a5492904b315`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:39:10.006115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f9b8b499-a14b-4dd3-b8b5-778e50bec454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:10.014330 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4388/8705] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:39:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:39:10.027356 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:39:10.029357 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:39:10.111026 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5a3c9-cdb6-486d-9e20-34766e1d8283 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:39:10.116696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:39:10.117321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:39:10.117579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:39:10.117847 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:39:10.123263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:39:10.128203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:39:10.130263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:39:10.141193 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4318/8706] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:39:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:39:19.999166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=17 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:39:20.135864 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:40:04.695101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:04.695655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:04.695655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:04.715267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] MFA Rules not processed for user `45c51ff8103f42fa9d68a5492904b315`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:40:04.716668 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:40:04.725197 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4389/8707] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:40:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:40:14.710406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:40:58.197884 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.211323 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d223a2c-2093-4684-9eef-cb564f1e3168 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.216829 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.217487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.217771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.218048 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] PATH_INFO: `/v3/users/45c51ff8103f42fa9d68a5492904b315` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.219202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c51ff8103f42fa9d68a5492904b315)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.221807 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.269120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c51ff8103f42fa9d68a5492904b315'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.269948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c51ff8103f42fa9d68a5492904b315'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.277103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c51ff8103f42fa9d68a5492904b315'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.284487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c51ff8103f42fa9d68a5492904b315'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.285161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5ecd82b5-3cb6-4ae6-b28a-d54fec4210c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c51ff8103f42fa9d68a5492904b315'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.303558 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4319/8708] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:40:58 2022] DELETE /identity/v3/users/45c51ff8103f42fa9d68a5492904b315 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:40:58.480689 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.500107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224689d-04c0-43fc-a67d-711cfdb441f4 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.558058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.558713 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.558989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.559260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] PATH_INFO: `/v3/projects/6a5a6205a2c74f07bda8d3bf7b5a5e6d` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.560364 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5a6205a2c74f07bda8d3bf7b5a5e6d)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.562249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.608616 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5a6205a2c74f07bda8d3bf7b5a5e6d', 'request_id': 'req-e49641d9-a167-4c82-bc03-f0f15ab6eea2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.614236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5a6205a2c74f07bda8d3bf7b5a5e6d', 'request_id': 'req-e49641d9-a167-4c82-bc03-f0f15ab6eea2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.620345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5a6205a2c74f07bda8d3bf7b5a5e6d', 'request_id': 'req-e49641d9-a167-4c82-bc03-f0f15ab6eea2'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.638186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] The token cache is being invalidate because project 6a5a6205a2c74f07bda8d3bf7b5a5e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:40:58.638481 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.639186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-e49641d9-a167-4c82-bc03-f0f15ab6eea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:40:58.640739 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4390/8709] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:40:58 2022] DELETE /identity/v3/projects/6a5a6205a2c74f07bda8d3bf7b5a5e6d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:40:58.646338 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.646672 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.646952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.666222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:40:58.667458 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:40:58.727311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4320/8710] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:40:58.736509 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.736804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.737116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.757492 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:40:58.759350 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:40:58.767366 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4391/8711] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:40:58.771742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.781449 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09d1d-ed31-4261-b6f2-9fa8f6258e5b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.786165 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.786799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.787086 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.787361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.788005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:40:58.788307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.790284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.802799 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4321/8712] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:40:58.807081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.817358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af272173-506a-48cf-ae17-e4871d09ad53 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.822148 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.822762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.823042 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.823308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.823923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.825837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.852888 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4392/8713] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:40:58 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:40:58.857099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.868655 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1053c0-2a8a-47b3-9f3c-bfb93db1fb71 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.927073 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.927700 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.927982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.928254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.928878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.930869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.938709 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:40:58.953227 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4322/8714] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:40:58 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:40:58.958139 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:58.968537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fccb7b-d057-4289-93ea-8f18f033bdb2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:58.973769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:58.974354 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:58.974675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:58.974950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:58.975584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:58.977168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:58.987532 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4393/8715] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:40:58.991534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:59.001774 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d7ea-71fe-4ba0-9b55-d01da775e6f8 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:59.006377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:59.007001 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:59.007282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:59.007553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:59.008211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:59.009799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea3869-f00b-4031-9466-63f5db58ada2 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:59.022047 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4323/8716] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:40:59.026152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:59.036927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb460322-0cbd-4615-9038-f17529512ecf admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:59.041557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:59.042216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:59.042522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:59.042932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] PATH_INFO: `/v3/projects/f1f293bffcad4e568dccab47bfc498aa/users/f9bf67f92b7142298f15e941a80a07d6/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:59.059471 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f293bffcad4e568dccab47bfc498aa, user_id=f9bf67f92b7142298f15e941a80a07d6, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:59.062463 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:59.076687 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4394/8717] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:40:59 2022] PUT /identity/v3/projects/f1f293bffcad4e568dccab47bfc498aa/users/f9bf67f92b7142298f15e941a80a07d6/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:40:59.082234 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b618-63da-488b-b406-f166ebd10298 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:59.082555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b618-63da-488b-b406-f166ebd10298 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:59.082830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b618-63da-488b-b406-f166ebd10298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:59.105652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8583b618-63da-488b-b406-f166ebd10298 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:40:59.106997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8583b618-63da-488b-b406-f166ebd10298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:40:59.169051 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4324/8718] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:40:59.179599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:40:59.181291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:40:59.212074 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:40:59.241039 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd11d6c-a1f2-4cb3-83f3-a5695690b65b admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:40:59.244836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:40:59.245445 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:40:59.245727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:40:59.246000 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:40:59.255088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:40:59.256516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:40:59.265939 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4395/8719] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:41:03.014224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e9fc-854b-4709-95be-647107bf823a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:41:03.014739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e9fc-854b-4709-95be-647107bf823a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:41:03.014831 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e9fc-854b-4709-95be-647107bf823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:41:03.044808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-cb51e9fc-854b-4709-95be-647107bf823a None None] MFA Rules not processed for user `f9bf67f92b7142298f15e941a80a07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:41:03.046019 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb51e9fc-854b-4709-95be-647107bf823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:41:03.137297 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4325/8720] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:41:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:41:03.152598 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:41:03.155125 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:41:03.270238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1d0e0-e686-490d-80e1-5fbfa904ad84 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:41:03.276197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:41:03.276817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:41:03.277100 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:41:03.277380 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:41:03.286649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:41:03.288258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566ed39-7484-4d14-888f-065a509f0f5e service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:41:03.297919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4396/8721] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:41:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:41:04.951285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:41:04.951739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:41:04.951974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:41:04.972810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] MFA Rules not processed for user `f9bf67f92b7142298f15e941a80a07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:41:04.973997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:41:04.981484 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4326/8722] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:41:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:41:13.293602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=60 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:41:14.966629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:42:23.003122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:42:23.003624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:42:23.003924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:42:23.023654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] MFA Rules not processed for user `f9bf67f92b7142298f15e941a80a07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:42:23.024876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:42:23.031859 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4397/8723] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:42:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:42:23.608099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:42:23.609552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:42:23.622065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efe40a-2364-453c-8e23-f2936daf6bb1 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:42:23.625666 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:42:23.626256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:42:23.626575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:42:23.626837 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:42:23.628499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:42:23.645275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:42:23.646803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:42:23.660263 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4327/8724] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:42:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:42:24.066274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:42:24.076937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a10bf9-3a2d-435e-a0ae-746d4cc44cb6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:42:24.080473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:42:24.081065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:42:24.081348 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:42:24.081639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:42:24.083487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:42:24.103056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:42:24.104696 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:42:24.119087 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4398/8725] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:42:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:42:26.550833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:42:26.570457 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35526c15-971f-4ddb-abc9-186553fbb760 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:42:26.576795 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:42:26.577552 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:42:26.577937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:42:26.578400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:42:26.580187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:42:26.595524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:42:26.597124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:42:26.611697 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4328/8726] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:42:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:42:34.111329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:42:36.604384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=12 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:43:11.383507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:11.392026 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e09b94-9753-4b0d-857f-3c1efb4dbd9e service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:11.395673 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:11.396263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:11.396543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:11.396815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:11.398558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:11.410703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:11.412124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:11.423328 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4399/8727] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:21.416611 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:43:24.672515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:24.673916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:24.746170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f8a-bbb0-43da-9522-970604f4953a service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:24.751219 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:24.751814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:24.752094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:24.752366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:24.754092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:24.768480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:24.770093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:24.783183 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4329/8728] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:27.352146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:27.353724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:27.372416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16756c6-61d5-4690-b3b4-c0428879aef8 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:27.377525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:27.378122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:27.378403 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:27.378706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:27.380427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:27.395193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:27.396773 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:27.409286 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4400/8729] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:34.775727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=32 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:43:37.402647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:43:59.493181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:59.503770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a4f2-336a-492c-b973-0ba17fb1ff7d service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:59.508791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:59.509510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:59.509742 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:59.510180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:59.512138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:59.524948 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:59.526965 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:59.538302 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4330/8730] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:59.642461 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:59.657068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12545e9-5d77-4474-bb36-d9893f298ef6 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:59.662084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:59.662708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:59.662981 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:59.663249 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:59.664954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:59.677279 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:59.678885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:59.688595 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4401/8731] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:59.866402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:59.875044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa3a8-4bef-4845-ba17-86a799fe38f6 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:59.878714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:59.879307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:59.879585 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:59.879860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:59.881571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:43:59.898277 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:43:59.899735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:43:59.913812 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4331/8732] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:43:59.975494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:43:59.988878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f253c-c169-4ca7-b21d-e0a571259b90 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:43:59.994092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:43:59.994712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:43:59.995012 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:43:59.995306 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:43:59.997603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:00.008945 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:44:00.010571 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:44:00.020919 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4402/8733] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:43:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:44:05.492296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:44:05.501212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdeea1-4e7b-4761-8d24-e8bdb4b80f75 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:44:05.505032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:44:05.505635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:44:05.505919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:44:05.506196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:44:05.507973 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:05.521142 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:44:05.522625 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11688e86-43e6-4b5e-b71a-8b8c9f27d9cc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:44:05.534958 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4332/8734] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:44:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:44:10.016943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:44:12.179989 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:44:12.181510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:12.206695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae6854-478b-4598-9338-f13c2b9af73d service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:44:12.212432 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:44:12.213067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:44:12.213349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:44:12.213621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:44:12.215389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:12.233166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:44:12.235195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:44:12.249590 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4403/8735] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:44:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:44:15.529906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=18 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:44:16.313016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:44:16.313304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:44:16.313584 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:44:16.333087 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] MFA Rules not processed for user `f9bf67f92b7142298f15e941a80a07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:44:16.334280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:16.341175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4333/8736] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:44:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:44:16.650072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:44:16.660053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82b1c7-c246-4220-b540-955e89497cc1 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:44:16.663692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:44:16.664281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:44:16.664560 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:44:16.664834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:44:16.666575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:16.679005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:44:16.680420 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45def68f-c6ee-47c9-b283-c61482dbe0f8 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:44:16.691560 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4404/8737] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:44:17.018456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:44:17.034091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4556-f8d5-4fa7-8658-f266a51c90be None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:44:17.039113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:44:17.039712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:44:17.039989 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:44:17.040257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:44:17.041946 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:44:17.054699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:44:17.056375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:44:17.066730 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4334/8738] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:44:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:44:26.684768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:44:27.062507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:45:27.693107 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829554-4a3f-4bb8-9944-4824f9209f09 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:45:27.693708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829554-4a3f-4bb8-9944-4824f9209f09 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:45:27.693708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829554-4a3f-4bb8-9944-4824f9209f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:45:27.725375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-bc829554-4a3f-4bb8-9944-4824f9209f09 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:45:27.726644 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-bc829554-4a3f-4bb8-9944-4824f9209f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:45:27.735538 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4405/8739] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 24 21:45:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:45:27.749760 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:45:27.763246 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a0b98-604b-4e0c-ba7e-1ea3cebd3576 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:45:27.769011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:45:27.769613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:45:27.769963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:45:27.770235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:45:27.778693 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:45:27.780269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0155c-fce6-459a-b4d8-b7b4d5314539 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:45:27.790974 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4335/8740] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:45:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:45:37.721150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=7 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:45:37.786616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:45:48.536704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:45:48.537242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:45:48.537242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:45:48.557915 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:45:48.559152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:45:48.568362 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4406/8741] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 24 21:45:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:45:48.579659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:45:48.580421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:45:48.580758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:45:48.600941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:45:48.602259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:45:48.611731 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4336/8742] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 21:45:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:45:48.617448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:45:48.629416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b84c2-715b-446e-adc2-b9bfbec5cae7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:45:48.635508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d502d-c236-47fc-8686-d46325371329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:45:48.637053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d502d-c236-47fc-8686-d46325371329 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:45:48.637337 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d502d-c236-47fc-8686-d46325371329 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:45:48.637657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d502d-c236-47fc-8686-d46325371329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:45:48.647507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d502d-c236-47fc-8686-d46325371329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:45:48.649095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d502d-c236-47fc-8686-d46325371329 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:45:48.666883 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4407/8743] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:45:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:45:58.595189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:45:58.655082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:46:04.176793 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:46:04.193247 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aef44-8941-48d8-b508-570a73d089e5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:46:04.198659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:46:04.199248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:46:04.199525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:46:04.199797 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:46:04.208190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:46:04.210632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:46:04.219768 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4337/8744] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:46:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:46:04.308299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:46:04.308759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:46:04.308860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:46:04.328256 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] MFA Rules not processed for user `f9bf67f92b7142298f15e941a80a07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:46:04.329467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:46:04.336308 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4408/8745] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:46:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:46:04.645284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:46:04.654613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700c423-b8d6-4b4f-8f9f-358ed677bae3 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:46:04.658175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:46:04.658805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:46:04.659633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:46:04.659925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:46:04.661590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:46:04.673664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:46:04.675819 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:46:04.686792 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4338/8746] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:46:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:46:14.323584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:46:14.680579 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=12 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:46:57.798836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:46:57.808868 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98fc496-cec7-46b8-8d89-75f782995f28 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:46:57.813013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:46:57.813612 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:46:57.813888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:46:57.814160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:46:57.823718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:46:57.825137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:46:57.835559 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4409/8747] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:00.756152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:00.768932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2b666-3ddd-4335-aec2-0d7caa697cbf service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:00.776122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:00.777041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:00.777627 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:00.778047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:00.779963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:00.796404 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:00.798182 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:00.813901 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4339/8748] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:47:00.837515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:00.849455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185d256-c030-4e29-82be-f89f64f57651 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:00.855639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:00.856270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:00.856545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:00.857198 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] PATH_INFO: `/v3/users/f9bf67f92b7142298f15e941a80a07d6` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:00.858642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bf67f92b7142298f15e941a80a07d6)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:00.861010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:00.919863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bf67f92b7142298f15e941a80a07d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:00.920521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bf67f92b7142298f15e941a80a07d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:00.928459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bf67f92b7142298f15e941a80a07d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:00.940561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bf67f92b7142298f15e941a80a07d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:00.941889 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-2ccf7085-7c2a-4bfe-8e5c-c2cdc7cedba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bf67f92b7142298f15e941a80a07d6'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:00.954262 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4410/8749] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:47:00 2022] DELETE /identity/v3/users/f9bf67f92b7142298f15e941a80a07d6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:01.147940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.158751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7b728-d276-458d-ab3e-783f11a3a567 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.224236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.224877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.225160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.225435 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] PATH_INFO: `/v3/projects/f1f293bffcad4e568dccab47bfc498aa` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.226594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f293bffcad4e568dccab47bfc498aa)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.228510 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.280952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f293bffcad4e568dccab47bfc498aa', 'request_id': 'req-bb1a2a81-69d7-48d9-8c02-57ede70c3330'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:01.287912 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f293bffcad4e568dccab47bfc498aa', 'request_id': 'req-bb1a2a81-69d7-48d9-8c02-57ede70c3330'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:01.295293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f293bffcad4e568dccab47bfc498aa', 'request_id': 'req-bb1a2a81-69d7-48d9-8c02-57ede70c3330'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:01.308141 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] The token cache is being invalidate because project f1f293bffcad4e568dccab47bfc498aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:47:01.308431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:01.309077 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-bb1a2a81-69d7-48d9-8c02-57ede70c3330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:01.310587 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4340/8750] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:47:01 2022] DELETE /identity/v3/projects/f1f293bffcad4e568dccab47bfc498aa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:01.315622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655101-2518-4292-8634-cd66f9892d85 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.315914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655101-2518-4292-8634-cd66f9892d85 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.316810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83655101-2518-4292-8634-cd66f9892d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.337275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-83655101-2518-4292-8634-cd66f9892d85 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:01.338510 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-83655101-2518-4292-8634-cd66f9892d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:01.396981 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4411/8751] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:47:01.404863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.405147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.405448 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.426805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:01.428257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:01.436124 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4341/8752] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:01.440312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83655101-2518-4292-8634-cd66f9892d85 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.450579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83655101-2518-4292-8634-cd66f9892d85 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.455157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.455748 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.456025 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.456294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.457768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:47:01.458144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.459808 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.476053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4412/8753] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:47:01.480237 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.489497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e35c8-6b22-4f4d-a105-965059bea465 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.494088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.494707 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.495013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.495283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.495896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.497784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.526246 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4342/8754] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:47:01 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:01.530600 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.540776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a167-3f21-4bcf-ab02-63dbee8153ed admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.585046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:01.600051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.600658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.600940 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.601210 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.601835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.603789 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.624219 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4413/8755] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:47:01 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:01.629265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.640534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06594a91-970d-45c1-a474-40acc750329e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.645940 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.646600 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.646882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.647264 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.647906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.649499 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.659390 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4343/8756] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:01.663623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.673535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44657940-a11e-4c3a-a4e4-3ad4bbf0d018 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.678920 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.679524 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.679803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.680099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.680758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.682394 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9979d2-5457-4da9-9c1e-eda328e1679a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.694038 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4414/8757] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:01.698093 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.709002 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624118f-9046-4681-a193-23fcb9d513cb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.713785 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.714427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.714736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.715017 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] PATH_INFO: `/v3/projects/cd01f24cc14c4cde8e764fc6a76c4944/users/11b872b8266f49e2b46706ef647ef4df/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.729373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd01f24cc14c4cde8e764fc6a76c4944, user_id=11b872b8266f49e2b46706ef647ef4df, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.732099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.744234 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4344/8758] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:47:01 2022] PUT /identity/v3/projects/cd01f24cc14c4cde8e764fc6a76c4944/users/11b872b8266f49e2b46706ef647ef4df/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:01.749284 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b280f-eee4-4d81-8a13-7c18bd722c17 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.749575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b280f-eee4-4d81-8a13-7c18bd722c17 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.749835 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b280f-eee4-4d81-8a13-7c18bd722c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.771223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4a7b280f-eee4-4d81-8a13-7c18bd722c17 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:01.772430 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4a7b280f-eee4-4d81-8a13-7c18bd722c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:01.831076 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4415/8759] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:01.841764 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:01.843614 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:01.860154 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:01.903805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e119ba-51df-467a-ab5b-b6168f3e7ac5 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:01.907483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:01.908083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:01.908364 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:01.908638 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:01.918306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:01.919791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db749d64-dc59-4efe-a0a5-48d71170176a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:01.929739 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4345/8760] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:06.078995 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:06.079525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:06.079525 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:06.111526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] MFA Rules not processed for user `11b872b8266f49e2b46706ef647ef4df`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:06.118128 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:06.186347 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4416/8761] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:06.196330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cef32-d1a0-4ea2-9515-0785c767f840 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:06.196889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cef32-d1a0-4ea2-9515-0785c767f840 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:06.197057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cef32-d1a0-4ea2-9515-0785c767f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:06.216818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-489cef32-d1a0-4ea2-9515-0785c767f840 None None] MFA Rules not processed for user `11b872b8266f49e2b46706ef647ef4df`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:06.218137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-489cef32-d1a0-4ea2-9515-0785c767f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:06.226224 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4346/8762] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:06.238591 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:06.240006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:06.297475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:06.321758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6352-d521-4fc4-978b-e2f04b1939ee None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:06.327332 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:06.327943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:06.328219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:06.328485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:06.337883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:06.339554 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:06.350276 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4417/8763] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:16.210949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=23 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:16.344765 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:16.544330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:16.544616 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:16.544892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:16.564317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] MFA Rules not processed for user `11b872b8266f49e2b46706ef647ef4df`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:16.565515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:16.572464 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4347/8764] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:24.132184 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:24.152197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466096-1613-4e86-a3e1-9a1755c3e201 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:24.155874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:24.156475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:24.156752 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:24.157023 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:24.158815 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:24.215666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:24.217090 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:24.231215 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4418/8765] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:26.559030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:26.583366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:26.595719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ba2db-4b72-42dd-8a4c-1f9cf216c00b None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:26.599371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:26.599957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:26.600235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:26.600504 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:26.602209 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:26.618808 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:26.620327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:26.634889 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4348/8766] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:47:28.746692 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:28.748494 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:28.778358 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3289e0-7d9a-4a5a-99da-73e63d455141 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:28.785375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:28.786067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:28.786335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:28.787290 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:28.788485 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:28.805450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:28.807101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:28.822314 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4419/8767] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:36.626665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=8 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:38.567978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:38.581575 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c2eb8-f67d-40ff-90dd-968b6828e719 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:38.586934 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:38.587569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:38.587852 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:38.588131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] PATH_INFO: `/v3/users/11b872b8266f49e2b46706ef647ef4df` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:38.589272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b872b8266f49e2b46706ef647ef4df)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:38.591269 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:38.644802 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b872b8266f49e2b46706ef647ef4df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.646180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b872b8266f49e2b46706ef647ef4df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.653406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b872b8266f49e2b46706ef647ef4df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.660354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b872b8266f49e2b46706ef647ef4df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.661594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-99bde571-c7da-4996-bc23-51cc3bcda3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b872b8266f49e2b46706ef647ef4df'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.679439 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4349/8768] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:47:38 2022] DELETE /identity/v3/users/11b872b8266f49e2b46706ef647ef4df => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:47:38.813702 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=34 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:38.856212 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:38.867429 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44056-7773-4ab7-91a7-82bb1a14d0a2 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:38.934486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:38.935143 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:38.935421 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:38.935721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] PATH_INFO: `/v3/projects/cd01f24cc14c4cde8e764fc6a76c4944` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:38.936841 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd01f24cc14c4cde8e764fc6a76c4944)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:38.938762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:38.985299 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd01f24cc14c4cde8e764fc6a76c4944', 'request_id': 'req-165c92fe-b4f7-484d-97c8-0e04a2f674ee'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.990832 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd01f24cc14c4cde8e764fc6a76c4944', 'request_id': 'req-165c92fe-b4f7-484d-97c8-0e04a2f674ee'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:38.997282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd01f24cc14c4cde8e764fc6a76c4944', 'request_id': 'req-165c92fe-b4f7-484d-97c8-0e04a2f674ee'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:39.022009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] The token cache is being invalidate because project cd01f24cc14c4cde8e764fc6a76c4944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:47:39.022283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:39.022933 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-165c92fe-b4f7-484d-97c8-0e04a2f674ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:39.024410 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4420/8769] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:47:38 2022] DELETE /identity/v3/projects/cd01f24cc14c4cde8e764fc6a76c4944 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:47:39.029396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.029730 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.029978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.059682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:39.060928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:39.119495 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4350/8770] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:39.127468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.127758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.128036 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.148547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:39.149760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:39.157663 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4421/8771] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:39.161882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.172660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fa0e4-43fd-448f-b834-063969a133bf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.178734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.179513 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.179835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.180140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.181031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:47:39.181337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.183936 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.196469 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4351/8772] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 21:47:39.200860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.211521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ff6bd-9746-42fc-ab74-705837fbc17d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.216289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.216886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.217168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.217441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.218068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.220013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.247553 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4422/8773] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:47:39 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:39.251863 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.262935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c47ed2-d051-4291-b258-825510c9a36b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.310044 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:39.322418 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.323051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.323331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.323605 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.324238 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.326169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.349112 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4352/8774] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:47:39 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:39.353572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.365561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b45bd-1068-4359-b65a-8e44dd2d7525 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.370441 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.371081 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.371383 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.371642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.372276 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.374235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.385253 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4423/8775] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:39.389322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.400951 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a00081-8a6d-4002-961c-9044a892bd0b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.405869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.406473 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.406773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.407046 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.407705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.409310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e312810-fdb9-4e32-89dd-a079330cea33 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.420435 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4353/8776] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:47:39.424459 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.435796 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0f64-06d4-4137-b6c4-9ce3635d8d51 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.440812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.441468 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.441747 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.442613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] PATH_INFO: `/v3/projects/8c958b302fc4432f965b4f4df2377ba5/users/e746fd2c7e524f4bb8be7b4553785958/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.458559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c958b302fc4432f965b4f4df2377ba5, user_id=e746fd2c7e524f4bb8be7b4553785958, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.461519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.475511 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4424/8777] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:47:39 2022] PUT /identity/v3/projects/8c958b302fc4432f965b4f4df2377ba5/users/e746fd2c7e524f4bb8be7b4553785958/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:39.485763 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb01d-4684-4472-b493-d9bf588de3b1 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.486047 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb01d-4684-4472-b493-d9bf588de3b1 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.486321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fb01d-4684-4472-b493-d9bf588de3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.514751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4e0fb01d-4684-4472-b493-d9bf588de3b1 None None] MFA Rules not processed for user `e746fd2c7e524f4bb8be7b4553785958`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:39.515954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4e0fb01d-4684-4472-b493-d9bf588de3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:39.579407 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4354/8778] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:39.589997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:39.591781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:39.601921 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:39.667922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602f47-4378-4d0a-9dfd-7519b5601779 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:39.671707 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:39.672315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:39.672599 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:39.672873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:39.682050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:39.683521 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:39.692821 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4425/8779] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:48.337866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:48.338416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:48.338416 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:48.400996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:48.404099 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:48.500895 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4355/8780] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:48.508041 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:48.521054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2a482-b1f4-4fce-9744-cbe37985190e service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:48.526342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:48.526977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:48.527260 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:48.527532 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:48.532454 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:48.553054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:48.554836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c00f6-d41e-4d35-8145-ab50f50a0917 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:48.570723 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4426/8781] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:50.379762 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:50.437696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:50.444337 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb187ade-255f-4887-b032-49af4b5e9314 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:50.449442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:50.450085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:50.450363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:50.450667 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] PATH_INFO: `/v3/users/e746fd2c7e524f4bb8be7b4553785958` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:50.451801 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e746fd2c7e524f4bb8be7b4553785958)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:50.453737 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:50.500248 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e746fd2c7e524f4bb8be7b4553785958'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.500815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e746fd2c7e524f4bb8be7b4553785958'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.507967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e746fd2c7e524f4bb8be7b4553785958'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.514853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e746fd2c7e524f4bb8be7b4553785958'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.515494 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e746fd2c7e524f4bb8be7b4553785958'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.532771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4356/8782] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:47:50 2022] DELETE /identity/v3/users/e746fd2c7e524f4bb8be7b4553785958 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:50.537950 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:50.538416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:50.538546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:50.565985 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:50.567486 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:50.628403 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4427/8783] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:50.638905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:50.640386 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:50.697756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df367395-ae2a-4e01-a3e0-0cbc699e5982 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:50.701409 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:50.702004 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:50.702283 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:50.702587 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:50.712311 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:50.713739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09693-280c-49cd-af4a-61612e7e59ad service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:50.724755 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4357/8784] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:50.891929 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:50.893572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:50.907322 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdbe41-9e15-4d2b-9931-c43e4a9b0c6f None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:50.912547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:50.913188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:50.913482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:50.913758 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] PATH_INFO: `/v3/projects/8c958b302fc4432f965b4f4df2377ba5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:50.915592 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c958b302fc4432f965b4f4df2377ba5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:50.917721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:50.967240 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c958b302fc4432f965b4f4df2377ba5', 'request_id': 'req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.973174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c958b302fc4432f965b4f4df2377ba5', 'request_id': 'req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.979484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c958b302fc4432f965b4f4df2377ba5', 'request_id': 'req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.993073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] The token cache is being invalidate because project 8c958b302fc4432f965b4f4df2377ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:47:50.993375 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.994007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66ed5fdf-7c09-4a0b-af4d-fb5d51bc0985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:50.995586 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4428/8785] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:47:50 2022] DELETE /identity/v3/projects/8c958b302fc4432f965b4f4df2377ba5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:51.000805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.001088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.001359 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.039612 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:51.040815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:51.099866 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4358/8786] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:51.107684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.107969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.108267 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.116617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:51.131703 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:51.132922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:51.141067 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4429/8787] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:51.146018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.150873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:51.157677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0845f-89d9-4774-a45f-9f27696e7aaf None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.163511 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.164103 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.164379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.164647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.165284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:47:51.166215 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.167857 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.179559 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4359/8788] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:47:51.183943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.193788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfbfa8-b3a0-4341-b637-4f6d5bca5a47 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.198462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.199092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.199370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.199643 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.200275 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.202192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.239907 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4430/8789] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:47:51 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:47:51.243903 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.254571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bde48c-2e96-44cc-b113-8539451e17fe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.310775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.311372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.311650 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.311923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.312548 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.314463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.336320 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4360/8790] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:47:51 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:51.340647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.351892 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbc0d-23cd-4221-86bb-dbb03033769e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.356853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.357558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.357906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.358293 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.359035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.360698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.372341 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4431/8791] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:51.376450 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.388091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887189-524d-4ed0-928b-f42a9c353a95 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.392886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.393712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.394000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.394274 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.395036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.396642 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59829d-2556-4908-bb6c-bf269dfbc0e6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.407915 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4361/8792] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:47:51.412171 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.422657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc01a-60c0-4910-9a74-350cc192fbb1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.428506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.429167 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.429448 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.429723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] PATH_INFO: `/v3/projects/83a85179750845b58b31a83af690c3c5/users/4fc501e3447147f0a3f3553ba02c726c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.445046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a85179750845b58b31a83af690c3c5, user_id=4fc501e3447147f0a3f3553ba02c726c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.447753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.461515 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4432/8793] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:47:51 2022] PUT /identity/v3/projects/83a85179750845b58b31a83af690c3c5/users/4fc501e3447147f0a3f3553ba02c726c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:51.472429 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.473371 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.473736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.501138 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] MFA Rules not processed for user `4fc501e3447147f0a3f3553ba02c726c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:51.502268 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:51.566247 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4362/8794] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:51.576467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:51.578311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:51.636317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051e903-a20b-4021-95d8-7cf992f222af admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:51.640073 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:51.640683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:51.640963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:51.641238 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:51.650425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:51.651884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1657839-2999-4bdf-b7b4-f9f3b7160b69 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:51.661186 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4433/8795] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:56.076535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:56.135038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:56.140069 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42875-5a51-43e5-9bfe-1b6290e1057e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:56.145835 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:56.146506 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.146804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.147080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] PATH_INFO: `/v3/users/4fc501e3447147f0a3f3553ba02c726c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.148343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc501e3447147f0a3f3553ba02c726c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:56.150308 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:56.198408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc501e3447147f0a3f3553ba02c726c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.199100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc501e3447147f0a3f3553ba02c726c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.206748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc501e3447147f0a3f3553ba02c726c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.213773 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc501e3447147f0a3f3553ba02c726c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.214401 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc501e3447147f0a3f3553ba02c726c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.232029 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4363/8796] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:47:56 2022] DELETE /identity/v3/users/4fc501e3447147f0a3f3553ba02c726c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:56.236522 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.236980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.237076 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.256770 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:56.257980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:56.317915 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4434/8797] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:56.327867 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:56.329317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:56.531583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079c658-c560-46d5-8c84-5de538a5e159 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:56.536179 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:56.536750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.537028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.537295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.547050 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:56.548487 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0ccb1-8cb5-4aa4-859e-030490c176cd service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:56.558818 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4364/8798] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:47:56.730924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:56.732484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:56.748009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb3b17-e9e2-4afd-aa38-0e99bb034b8a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:56.758182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:56.759345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.759537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.759818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] PATH_INFO: `/v3/projects/83a85179750845b58b31a83af690c3c5` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.761698 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a85179750845b58b31a83af690c3c5)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:56.763934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:56.799045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:56.816793 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a85179750845b58b31a83af690c3c5', 'request_id': 'req-88f1c74d-7630-4953-93b8-425a71ada928'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.822533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a85179750845b58b31a83af690c3c5', 'request_id': 'req-88f1c74d-7630-4953-93b8-425a71ada928'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.828658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a85179750845b58b31a83af690c3c5', 'request_id': 'req-88f1c74d-7630-4953-93b8-425a71ada928'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.840858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] The token cache is being invalidate because project 83a85179750845b58b31a83af690c3c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:47:56.841146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.841771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-88f1c74d-7630-4953-93b8-425a71ada928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:47:56.843356 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4435/8799] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:47:56 2022] DELETE /identity/v3/projects/83a85179750845b58b31a83af690c3c5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:56.849933 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.850212 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.850480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.870623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:56.871747 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:56.931326 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4365/8800] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:56.939438 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.939729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.940010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.960886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:56.962101 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:56.970075 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4436/8801] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:47:56.974632 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:56.984301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171d919-131a-4951-a3f9-59be13a35c3e None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:56.989065 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:56.989649 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:56.989928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:56.990200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:56.990868 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:47:56.991170 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:56.992751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.004993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4366/8802] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:47:57.009457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.018828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286fd26-50dd-454c-aa1f-f9b0c37cb981 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.024315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.024906 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.025185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.025458 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.026084 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.028016 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.054441 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4437/8803] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:47:57 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:57.058750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.069136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c8c68-26c2-4beb-99cb-f67be7a8f082 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.073082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:47:57.128578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.129699 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.129909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.130177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.130832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.132761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.153681 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4367/8804] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:47:57 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:47:57.158005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.168756 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d4918-a883-478a-89e1-9100efd291a5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.173719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.174335 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.174664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.174938 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.176077 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.177666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.187598 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4438/8805] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:57.192128 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.202092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95120b73-faa1-4f93-b3be-ed9d8d6eb952 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.206803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.207979 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.208186 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.208460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.209085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.210689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64346f0e-ba0b-41f4-8c25-ec98559c8010 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.220304 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4368/8806] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:47:57.225055 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.235336 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cb8b0-28ad-49cb-8972-ac9cbff1c9a3 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.240498 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.241133 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.241408 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.241679 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] PATH_INFO: `/v3/projects/ef4f79cc25ee473bb029c30cbf0da4a2/users/d6fb8f277c5047c7860690a102ef7c41/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.256579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4f79cc25ee473bb029c30cbf0da4a2, user_id=d6fb8f277c5047c7860690a102ef7c41, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.259243 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.271964 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4439/8807] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:47:57 2022] PUT /identity/v3/projects/ef4f79cc25ee473bb029c30cbf0da4a2/users/d6fb8f277c5047c7860690a102ef7c41/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:47:57.277101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536d04a-6dba-4d6e-9375-de85d181af79 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.277529 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536d04a-6dba-4d6e-9375-de85d181af79 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.277805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536d04a-6dba-4d6e-9375-de85d181af79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.296928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f536d04a-6dba-4d6e-9375-de85d181af79 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:47:57.298130 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f536d04a-6dba-4d6e-9375-de85d181af79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:57.357620 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4369/8808] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:47:57.367870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:47:57.369618 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:47:57.427466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315be578-73cb-4e08-b3ac-72cf3514ea35 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:47:57.431152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:47:57.431777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:47:57.432054 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:47:57.432349 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:47:57.441885 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:47:57.443401 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:47:57.452819 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4440/8809] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:01.440916 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.441728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.442113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.471757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] MFA Rules not processed for user `d6fb8f277c5047c7860690a102ef7c41`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:01.477152 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:01.562678 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4370/8810] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:48:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:01.566954 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.578433 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859e724-6545-4607-91ed-1fbed1d731ab service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.583785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.584399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.584701 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.584974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.585605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.587579 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.624732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4441/8811] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:48:01 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:48:01.628309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.641806 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75c0c4-5298-4141-9dfc-ee2c4d1c2934 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.654373 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:01.721198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.721721 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.722000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.722296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.722961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.724873 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.753671 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4371/8812] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:48:01 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:48:01.758281 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.770857 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b6864-9fc7-429b-a1af-10845b47f291 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.775813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.776533 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.776893 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.777227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.777931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.779720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.801475 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4442/8813] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:48:01.803084 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.813937 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce9f06-c7b0-4111-9a2d-1cd721f415d6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.821615 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.823336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.823714 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.823991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] PATH_INFO: `/v3/projects/115d7ae5f9114c21b292b0fafb8b5564/users/788af0021f014a00a501aab4bef1146e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.842561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115d7ae5f9114c21b292b0fafb8b5564, user_id=788af0021f014a00a501aab4bef1146e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.845391 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.859342 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4372/8814] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:48:01 2022] PUT /identity/v3/projects/115d7ae5f9114c21b292b0fafb8b5564/users/788af0021f014a00a501aab4bef1146e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:48:01.863956 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.874961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed69bc-b6d2-4576-aadf-a6d873ba8dc2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.896246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:01.940236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.940785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.941121 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.941910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.942617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.944274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476e475-55d3-485f-b8b6-664ff9e47add admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.955027 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4443/8815] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:48:01.960305 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:01.971061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375217b-ed2f-4da1-bf51-aec23d5123a4 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:01.977443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:01.978035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:01.978307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:01.978647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] PATH_INFO: `/v3/projects/115d7ae5f9114c21b292b0fafb8b5564/users/788af0021f014a00a501aab4bef1146e/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:01.980557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=115d7ae5f9114c21b292b0fafb8b5564, user_id=788af0021f014a00a501aab4bef1146e, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:01.983233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3ab4-59bd-4049-87ac-9251f8bd61bf admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:01.995178 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4373/8816] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:48:01 2022] PUT /identity/v3/projects/115d7ae5f9114c21b292b0fafb8b5564/users/788af0021f014a00a501aab4bef1146e/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:48:06.459395 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:06.459946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:06.459946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:06.490310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] MFA Rules not processed for user `788af0021f014a00a501aab4bef1146e`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:06.491567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:06.565881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4444/8817] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:48:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:06.602155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b9b2-35ca-421e-a423-5d07d918fea7 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:06.602641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b9b2-35ca-421e-a423-5d07d918fea7 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:06.602804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141b9b2-35ca-421e-a423-5d07d918fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:06.671110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-1141b9b2-35ca-421e-a423-5d07d918fea7 None None] MFA Rules not processed for user `99f0d8a1aa0d4bacbd4e4fb2e7864cee`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:06.672437 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-1141b9b2-35ca-421e-a423-5d07d918fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:06.744636 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4374/8818] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 21:48:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:06.751085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:06.761536 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c9c16-6632-4ade-8858-f02ab0fb4e79 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:06.765565 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:06.766182 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:06.766457 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:06.766759 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:06.852455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:06.853902 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1feb4-fa7b-4669-864a-c2207a510e7d service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:06.866970 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4445/8819] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:07.027379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:07.027770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:07.028146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:07.082975 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:07.084318 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:07.101772 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:07.157930 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4375/8820] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:07.208676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaeeaf-ab84-4222-b2d3-59175597242a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:07.208955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaeeaf-ab84-4222-b2d3-59175597242a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:07.209327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaeeaf-ab84-4222-b2d3-59175597242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:07.266177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-cfcaeeaf-ab84-4222-b2d3-59175597242a None None] MFA Rules not processed for user `0b5a7880d6ec47f381af9ef1f09a767c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:07.267515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-cfcaeeaf-ab84-4222-b2d3-59175597242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:07.280160 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=100 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:07.341340 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4446/8821] 10.222.0.52 () {58 vars in 1040 bytes} [Sat Sep 24 21:48:07 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:07.347702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:07.359591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fb418-9a4f-4bcc-aa21-6007aabdc4a5 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:07.365256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:07.365959 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:07.366120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:07.366411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:07.375236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:07.377180 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adffa7-f811-45a2-8af3-e7a1d706a469 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:07.386318 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4376/8822] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:48:16.614152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d781b-a45a-41b2-8735-1ff8f925a83b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:16.614664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d781b-a45a-41b2-8735-1ff8f925a83b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:16.614762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d781b-a45a-41b2-8735-1ff8f925a83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:16.635753 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-496d781b-a45a-41b2-8735-1ff8f925a83b None None] MFA Rules not processed for user `d6fb8f277c5047c7860690a102ef7c41`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:16.636967 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-496d781b-a45a-41b2-8735-1ff8f925a83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:16.644602 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4447/8823] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:48:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:16.676943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:16.677710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:16.677805 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:16.715532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:16.716736 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:16.783004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4377/8824] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 21:48:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:16.789168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b795-bca3-4d03-a303-fcae0e19080d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:16.789451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b795-bca3-4d03-a303-fcae0e19080d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:16.789724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b795-bca3-4d03-a303-fcae0e19080d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:16.810150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-98b9b795-bca3-4d03-a303-fcae0e19080d None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:16.811384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-98b9b795-bca3-4d03-a303-fcae0e19080d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:16.821313 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4448/8825] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:16.833444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:16.842677 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fb1f4-5970-40b9-8899-f0b688a2379c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:16.846228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:16.846977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:16.847279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:16.847559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:16.857582 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:16.859030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:16.870820 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4378/8826] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:17.094998 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548e6-5445-44cc-9566-8c39c8d3e595 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:17.095292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548e6-5445-44cc-9566-8c39c8d3e595 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:17.095568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548e6-5445-44cc-9566-8c39c8d3e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:17.134671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-029548e6-5445-44cc-9566-8c39c8d3e595 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:17.135883 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-029548e6-5445-44cc-9566-8c39c8d3e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:17.195338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4449/8827] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:17.206883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:17.208272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:17.219935 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462337a-a372-4f0a-837c-672ee7adf771 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:17.224021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:17.224599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:17.224877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:17.225148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:17.232129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:17.233583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d4afa-6968-4537-acd6-3ab85b37abc5 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:17.241915 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4379/8828] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:17.581894 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:17.582399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:17.582769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:17.603200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:17.604318 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:17.614593 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4450/8829] 10.222.0.52 () {58 vars in 1058 bytes} [Sat Sep 24 21:48:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:17.620711 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213af664-4e38-4b17-ae8c-91b702bc14f3 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:17.620997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213af664-4e38-4b17-ae8c-91b702bc14f3 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:17.621276 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213af664-4e38-4b17-ae8c-91b702bc14f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:17.643190 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-213af664-4e38-4b17-ae8c-91b702bc14f3 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:17.645240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-213af664-4e38-4b17-ae8c-91b702bc14f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:17.654454 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4380/8830] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:17.666670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:17.676908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9572866-7b55-4d73-b178-45fc66b5d4b3 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:17.681382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:17.682141 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:17.682607 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:17.682895 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:17.693731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:17.695196 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:17.707163 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4451/8831] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:48:18.893088 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20945-feeb-4f51-89bd-70db428309cc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:18.893663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20945-feeb-4f51-89bd-70db428309cc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:18.894298 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb20945-feeb-4f51-89bd-70db428309cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:18.914782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-9fb20945-feeb-4f51-89bd-70db428309cc None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:18.916132 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9fb20945-feeb-4f51-89bd-70db428309cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:18.923529 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4381/8832] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:18.935942 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:18.948683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5504f-4631-4459-a1fa-e8d0314675c7 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:18.953722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:18.954312 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:18.954617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:18.954886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:18.964045 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:18.965798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:18.974979 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4452/8833] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:48:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:48:26.639438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45c60-fca2-4b57-a14c-19697e45dcd2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:26.639925 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45c60-fca2-4b57-a14c-19697e45dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:26.640030 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c45c60-fca2-4b57-a14c-19697e45dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:26.659735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b7c45c60-fca2-4b57-a14c-19697e45dcd2 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:26.660929 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b7c45c60-fca2-4b57-a14c-19697e45dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:26.667377 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4382/8834] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 24 21:48:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:26.679846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:26.689118 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef2d2-602b-4e00-80a0-f30c310b42cf service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:26.693449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:26.694065 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:26.694344 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:26.694652 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:26.701621 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:26.703087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:26.710881 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4453/8835] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:27.435743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7e408-3559-4335-94dc-781ab01c1b8e None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:27.436033 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7e408-3559-4335-94dc-781ab01c1b8e None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:27.436307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7e408-3559-4335-94dc-781ab01c1b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:27.457932 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0ac7e408-3559-4335-94dc-781ab01c1b8e None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:27.459174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0ac7e408-3559-4335-94dc-781ab01c1b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:27.468453 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4383/8836] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:48:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:27.484684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:27.498363 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853ab2-377b-4920-bc90-0ad948a8f8c6 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:27.504944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:27.505551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:27.505830 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:27.506099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:27.515827 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:27.517406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:27.529131 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4454/8837] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:48:31.362909 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a38dcb-5fc5-4f26-a5eb-b2b8d2db084d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:31.363434 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a38dcb-5fc5-4f26-a5eb-b2b8d2db084d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:31.363543 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a38dcb-5fc5-4f26-a5eb-b2b8d2db084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:31.388981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-a7a38dcb-5fc5-4f26-a5eb-b2b8d2db084d None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:31.390177 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-a7a38dcb-5fc5-4f26-a5eb-b2b8d2db084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:31.400315 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4384/8838] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:48:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:31.417708 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:31.429328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08a8d6-c377-49b8-bce2-7c9657beb422 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:31.434978 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:31.435581 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:31.435880 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:31.436136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:31.445844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:31.447452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:31.458354 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4455/8839] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:48:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:48:38.857651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52bb2-f835-4731-8fb4-b2a4ea021c72 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:38.858101 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52bb2-f835-4731-8fb4-b2a4ea021c72 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:38.858199 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db52bb2-f835-4731-8fb4-b2a4ea021c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:38.866169 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=91 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:38.879622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-6db52bb2-f835-4731-8fb4-b2a4ea021c72 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:38.880815 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-6db52bb2-f835-4731-8fb4-b2a4ea021c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:38.888232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4385/8840] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:48:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:38.900900 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:38.906896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=98 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:38.913552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f8ba-c86a-41b0-8f64-7399cf41a848 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:38.918804 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:38.919386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:38.919659 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:38.919926 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:38.928655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:38.930218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:38.939057 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4456/8841] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:48:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:48:39.415768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cabf0-5e5d-4747-bc71-a63f2480d20d None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:39.416052 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cabf0-5e5d-4747-bc71-a63f2480d20d None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:39.416326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cabf0-5e5d-4747-bc71-a63f2480d20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:39.436546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b34cabf0-5e5d-4747-bc71-a63f2480d20d None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:39.437739 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b34cabf0-5e5d-4747-bc71-a63f2480d20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:39.445988 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4386/8842] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:39.459700 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:39.470165 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a83a0-099c-4309-bac5-c8a7ea3f87e1 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:39.475424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:39.476007 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:39.476283 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:39.476551 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:39.484464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:39.486038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb464d-7a00-47a2-8f09-cb8b3ba2a3f5 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:39.495429 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4457/8843] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:48:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:48:47.973817 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:47.974314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:47.974408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:47.993463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] MFA Rules not processed for user `d6fb8f277c5047c7860690a102ef7c41`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:47.994695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:48.001466 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4387/8844] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:48:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:49.492013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=11 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:49.843941 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff376-a5f1-4f6d-816d-700f3bca7148 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:49.844302 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff376-a5f1-4f6d-816d-700f3bca7148 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:49.844665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff376-a5f1-4f6d-816d-700f3bca7148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:49.866051 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-303ff376-a5f1-4f6d-816d-700f3bca7148 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:48:49.867422 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-303ff376-a5f1-4f6d-816d-700f3bca7148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:48:49.874749 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4458/8845] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:48:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:48:49.888075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:48:49.900507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ae8f1-97c5-4a5e-a54a-941ad39eda76 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:48:49.905715 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:48:49.906314 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:48:49.907217 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:48:49.907488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:48:49.917012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:48:49.918639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:48:49.927778 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4388/8846] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:48:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:48:59.860777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:48:59.923432 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=14 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:00.599258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c48a0-d759-4897-b2d8-270a2d80a275 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:00.599638 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c48a0-d759-4897-b2d8-270a2d80a275 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:00.600098 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c48a0-d759-4897-b2d8-270a2d80a275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:00.621976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c76c48a0-d759-4897-b2d8-270a2d80a275 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:00.623214 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c76c48a0-d759-4897-b2d8-270a2d80a275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:00.631157 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4459/8847] 10.222.0.25 () {58 vars in 1015 bytes} [Sat Sep 24 21:49:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:00.644759 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:00.654174 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cb83a-86e6-4e26-8f75-9913241bddf6 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:00.657770 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:00.658354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:00.659166 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:00.659441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:00.666733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:00.668161 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:00.677993 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4389/8848] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:49:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:49:10.617129 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:10.672637 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:16.878811 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951e15-8c76-4eea-9f9a-7d39f5deca7d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:16.879268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951e15-8c76-4eea-9f9a-7d39f5deca7d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:16.879359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7951e15-8c76-4eea-9f9a-7d39f5deca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:16.899225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-a7951e15-8c76-4eea-9f9a-7d39f5deca7d None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:16.900360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a7951e15-8c76-4eea-9f9a-7d39f5deca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:16.907550 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4460/8849] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 21:49:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:16.930733 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:16.943745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4337-0c28-4d2c-92a7-95fe376c3525 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:16.948683 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:16.949265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:16.949538 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:16.949804 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:16.958547 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:16.960129 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:16.967999 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4390/8850] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 21:49:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 21:49:20.176907 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:20.178274 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:20.178603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:20.253515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:20.254834 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:20.349732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4461/8851] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:49:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:20.360758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:20.362891 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:20.378082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bbdf5-7bf4-479d-9345-2f030be0b777 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:20.383810 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:20.384415 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:20.384698 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:20.384969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:20.394717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:20.396259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:20.406753 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4391/8852] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:49:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:49:23.525122 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:23.601478 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a9178-6915-4bee-8b6f-355fd5840f52 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:23.606168 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:23.606836 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:23.607116 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:23.607381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] PATH_INFO: `/v3/users/d6fb8f277c5047c7860690a102ef7c41` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:23.608504 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fb8f277c5047c7860690a102ef7c41)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:23.610424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:23.660917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fb8f277c5047c7860690a102ef7c41'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:23.661593 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fb8f277c5047c7860690a102ef7c41'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:23.664137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:23.669743 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fb8f277c5047c7860690a102ef7c41'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:23.683361 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fb8f277c5047c7860690a102ef7c41'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:23.684031 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fb8f277c5047c7860690a102ef7c41'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:23.696721 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4462/8853] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:49:23 2022] DELETE /identity/v3/users/d6fb8f277c5047c7860690a102ef7c41 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:23.877982 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:23.889014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809780d-2887-4e86-b834-52b3f73c79c4 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:23.952609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:23.953228 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:23.953512 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:23.953789 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] PATH_INFO: `/v3/projects/ef4f79cc25ee473bb029c30cbf0da4a2` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:23.955514 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4f79cc25ee473bb029c30cbf0da4a2)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:23.957595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.004309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4f79cc25ee473bb029c30cbf0da4a2', 'request_id': 'req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.009976 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4f79cc25ee473bb029c30cbf0da4a2', 'request_id': 'req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.017280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4f79cc25ee473bb029c30cbf0da4a2', 'request_id': 'req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.028599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] The token cache is being invalidate because project ef4f79cc25ee473bb029c30cbf0da4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:49:24.028886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.029502 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.031004 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4392/8854] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:49:23 2022] DELETE /identity/v3/projects/ef4f79cc25ee473bb029c30cbf0da4a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:24.035506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.036934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:24.051056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0b773-cd7e-4725-a722-0ea55e3dafd6 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.108559 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.109192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.109469 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.110017 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] PATH_INFO: `/v3/users/788af0021f014a00a501aab4bef1146e` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.112078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788af0021f014a00a501aab4bef1146e)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.113999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.161406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788af0021f014a00a501aab4bef1146e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.162057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788af0021f014a00a501aab4bef1146e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.169388 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788af0021f014a00a501aab4bef1146e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.181910 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788af0021f014a00a501aab4bef1146e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.182578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-13a234f4-33e3-4d3c-bf32-cf8309cb1906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788af0021f014a00a501aab4bef1146e'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.195208 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4463/8855] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:49:24 2022] DELETE /identity/v3/users/788af0021f014a00a501aab4bef1146e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:24.366651 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.368095 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:24.386828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60b2d-4166-49bb-bcab-8f1d33add9b6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.442664 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.443303 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.443589 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.443890 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] PATH_INFO: `/v3/projects/115d7ae5f9114c21b292b0fafb8b5564` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.445022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] RBAC: Authorizing `identity:delete_project(project_id=115d7ae5f9114c21b292b0fafb8b5564)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.446966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.453304 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:24.495607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115d7ae5f9114c21b292b0fafb8b5564', 'request_id': 'req-fe84070a-2fee-481c-82a1-f3cb16a4650e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.500958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115d7ae5f9114c21b292b0fafb8b5564', 'request_id': 'req-fe84070a-2fee-481c-82a1-f3cb16a4650e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.508665 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115d7ae5f9114c21b292b0fafb8b5564', 'request_id': 'req-fe84070a-2fee-481c-82a1-f3cb16a4650e'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.519834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] The token cache is being invalidate because project 115d7ae5f9114c21b292b0fafb8b5564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:49:24.520126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.520754 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-fe84070a-2fee-481c-82a1-f3cb16a4650e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:49:24.522232 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4393/8856] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:49:24 2022] DELETE /identity/v3/projects/115d7ae5f9114c21b292b0fafb8b5564 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:24.527366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.527852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.527951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.547516 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:24.549329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:24.593690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:24.613417 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4464/8857] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:24.621005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.621288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.621559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.643342 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:24.644549 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:24.652473 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4394/8858] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:24.656721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.667574 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16209e-edf2-4bfe-a1c0-02effaf64d08 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.672174 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.672746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.673021 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.673342 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.674062 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:49:24.674356 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.675963 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.687546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4465/8859] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:49:24.692563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.701824 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34b8ff-839b-4742-aba1-4a9d2c8be939 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.707611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.708221 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.708496 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.709061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.709678 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.711626 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.741104 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4395/8860] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:49:24 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:49:24.745435 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.755923 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be30e4-04d3-45d7-a287-5b960abe077e admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.814009 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.814862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.815138 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.815409 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.816032 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.818176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.840053 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4466/8861] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:49:24 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 21:49:24.844299 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.855331 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2bef0-babe-4cd1-8b77-596185757036 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.859911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.860491 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.861380 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.861858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.862676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.864320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.874908 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4396/8862] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:49:24.880013 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.890135 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d62ef-e7bd-4b81-8f3e-5663028ba6b0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.895222 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.895814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.896087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.896359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.896990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.898655 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225aaa-97f6-48e9-913e-2962263538be admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.910684 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4467/8863] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:49:24.914889 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:24.925470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72280237-91c0-46e0-9549-2e836979bbc0 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:24.930351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:24.931042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.931322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.931597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] PATH_INFO: `/v3/projects/22a8ea7438b641139afc6ea3c2dcba4c/users/ce56b96d91fa42029081bd0e6eccb5e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:24.946294 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a8ea7438b641139afc6ea3c2dcba4c, user_id=ce56b96d91fa42029081bd0e6eccb5e4, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:24.948980 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:24.961952 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4397/8864] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:49:24 2022] PUT /identity/v3/projects/22a8ea7438b641139afc6ea3c2dcba4c/users/ce56b96d91fa42029081bd0e6eccb5e4/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:24.973384 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:24.973722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:24.973999 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.002828 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:25.004075 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:25.068073 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4468/8865] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:49:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:49:25.072644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.135010 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa228d59-5655-4527-ad77-d362253c3080 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.139528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.140113 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.140385 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.140652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.141257 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.143553 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.169438 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4398/8866] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:49:25 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:49:25.174682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.185484 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259ef41-9bcb-46e8-9672-e9ac4e867ad6 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.226464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:25.243853 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.244451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.244727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.245001 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.245623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.247578 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.269016 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4469/8867] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:49:25 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:49:25.273240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.284343 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e734903-1874-460b-a7b2-5e298a106660 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.289008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.289599 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.289879 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.290153 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.290830 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.292420 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.303771 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4399/8868] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:49:25.307934 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.318467 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70853545-fe93-467f-967d-d037f4803d33 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.323176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.323817 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.324095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.324368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] PATH_INFO: `/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb/users/3ce7e2cd8de64a86827b47417a26c7aa/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.339472 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7fcf14adabe4ef2b3d9094cb80395cb, user_id=3ce7e2cd8de64a86827b47417a26c7aa, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.342105 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.355792 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4470/8869] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:49:25 2022] PUT /identity/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb/users/3ce7e2cd8de64a86827b47417a26c7aa/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:25.359752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.367013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:25.370972 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a4e87-f2b4-4132-aeae-f87eb99909e1 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.428998 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.429594 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.429963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.430233 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.430894 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.432478 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80bd11-95a1-4f6b-aaa1-9d255d750283 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.442674 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4400/8870] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:49:25.447030 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.458087 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f340e-dd41-4ff0-be4b-17d0c16bea36 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.462728 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.463365 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.463640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.463937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] PATH_INFO: `/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb/users/3ce7e2cd8de64a86827b47417a26c7aa/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.465826 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7fcf14adabe4ef2b3d9094cb80395cb, user_id=3ce7e2cd8de64a86827b47417a26c7aa, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.468507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.478521 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4471/8871] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:49:25 2022] PUT /identity/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb/users/3ce7e2cd8de64a86827b47417a26c7aa/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:49:25.488193 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.488505 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.488780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.516781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] MFA Rules not processed for user `3ce7e2cd8de64a86827b47417a26c7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:25.518623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:25.586564 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4401/8872] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:49:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:25.601411 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:25.603197 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:25.679172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a4559-bf21-4bdb-a885-ebab6d35bdc0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:25.686353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:25.687066 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:25.687292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:25.687642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:25.750722 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:25.752477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717d3fc-32eb-4d70-8a87-bef1ecd5be3e service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:25.761324 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4472/8873] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:49:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:49:28.481928 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:28.483471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:28.544275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2567ff-04a9-45c2-9025-46bc7b5d908a None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:28.548301 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:28.548920 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:28.549194 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:28.549471 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:28.559340 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:28.560782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:28.572141 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4402/8874] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:49:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:49:31.486566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:31.487067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:31.487172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:31.509986 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:49:31.513797 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:31.521879 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4473/8875] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:49:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:49:32.296007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:49:32.297545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:32.342092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=100 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:32.420460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfbf99-184b-49b4-bcb9-9fce82bfa5ac service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:49:32.426144 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:49:32.426943 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:49:32.427588 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:49:32.428079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:49:32.430654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:49:32.451406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:49:32.453060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7fd3d-a4ab-427d-acd2-6291ef8991cd service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:49:32.466624 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4403/8876] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:49:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:49:41.500852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:49:42.461119 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=19 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:50:32.659096 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:50:32.671512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2e59-6e55-4005-8831-27271102f882 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:50:32.677558 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:50:32.678188 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:50:32.678515 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:50:32.678783 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:50:32.680483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:50:32.692019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:50:32.693740 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:50:32.703402 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4474/8877] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:50:42.699360 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:50:53.911011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4440-c684-4de7-b98b-6ede6409d7f0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:50:53.911011 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4440-c684-4de7-b98b-6ede6409d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:50:53.911503 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4440-c684-4de7-b98b-6ede6409d7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:50:53.932902 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-40ec4440-c684-4de7-b98b-6ede6409d7f0 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:50:53.934297 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-40ec4440-c684-4de7-b98b-6ede6409d7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:50:53.944874 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4404/8878] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:50:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:50:53.957506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:50:53.959142 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:50:53.974705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec41a93-1be8-4825-b23d-11bcac2addd2 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:50:53.981088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:50:53.981782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:50:53.982583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:50:53.982583 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:50:53.993221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:50:53.994968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:50:54.007702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4475/8879] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:50:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:51:03.926156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:51:04.001856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:51:06.699057 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:51:06.699564 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:51:06.701216 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:51:06.724652 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:51:06.726143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:51:06.733941 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4405/8880] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:51:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:51:07.761766 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:51:07.776124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1d456-aa03-454a-a2d1-2732084eb46d service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:51:07.781412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:51:07.782195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:51:07.782483 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:51:07.782803 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:51:07.785035 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:51:07.798425 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:51:07.800060 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87e720-e623-45a2-b90b-ee700ee6b57c service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:51:07.817042 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4476/8881] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:51:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:51:08.398607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:51:08.400145 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:51:08.415014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a565ec-5ae0-423a-ae2c-4820c037f4dc None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:51:08.420713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:51:08.421320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:51:08.421604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:51:08.421877 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:51:08.423591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:51:08.435629 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:51:08.437317 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:51:08.448265 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4406/8882] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:51:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:51:17.812462 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:51:18.444085 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:52:45.773697 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f34d-33eb-49cf-a06f-36d8e1901f50 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:52:45.774286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f34d-33eb-49cf-a06f-36d8e1901f50 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:52:45.774286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f34d-33eb-49cf-a06f-36d8e1901f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:52:45.795685 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-28d9f34d-33eb-49cf-a06f-36d8e1901f50 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:52:45.797389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-28d9f34d-33eb-49cf-a06f-36d8e1901f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:52:45.807434 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4477/8883] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:52:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:52:45.833008 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:52:45.847481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f4bfc-09c4-4453-8986-af26e739080f service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:52:45.853524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:52:45.854240 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:52:45.854670 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:52:45.855042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:52:45.865124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:52:45.867192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3718a75-4e6d-439d-a169-96b56d9bdfcd service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:52:45.879375 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4407/8884] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:52:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:52:51.562543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:52:51.563464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:52:51.563818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:52:51.589231 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:52:51.590705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:52:51.598280 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4478/8885] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:52:51.973858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485363f-fb0c-4862-9991-7d507ff235d0 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:52:51.974355 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485363f-fb0c-4862-9991-7d507ff235d0 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:52:51.974641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485363f-fb0c-4862-9991-7d507ff235d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:52:51.995136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-f485363f-fb0c-4862-9991-7d507ff235d0 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:52:51.996442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f485363f-fb0c-4862-9991-7d507ff235d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:52:52.003676 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4408/8886] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:52:52.016389 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:52:52.017905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:52:52.030309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f1ce6-1165-475f-a193-847f9d916005 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:52:52.034108 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:52:52.034751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:52:52.035034 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:52:52.035308 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:52:52.043935 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:52:52.045381 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:52:52.054130 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4479/8887] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:52:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:53:01.989782 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=9 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:53:02.049858 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:54:18.237696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.238278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.238278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.258601 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] MFA Rules not processed for user `0b98deaef6c24ed2aa7c7ecc90c75205`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:54:18.259953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:18.260538 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:18.266536 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4409/8888] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:54:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:54:18.269527 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ebf5c-fb74-40c6-ac24-fba9945693b4 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:18.273294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:18.273918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.274193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.274473 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.276234 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:18.279406 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:18.288704 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a1945-645f-4fa1-8dce-16bd17c09d12 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:18.292551 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:18.293149 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.293431 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.293734 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.301211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:18.303641 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d360aa-2869-49b2-8afa-1cfbde9f08b3 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:18.311693 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:18.312089 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4410/8889] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:54:18.313209 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:18.335104 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4480/8890] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:54:18.426140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed10e48-0ef5-4200-8a5a-075ff239e0d8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.426445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed10e48-0ef5-4200-8a5a-075ff239e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.426719 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed10e48-0ef5-4200-8a5a-075ff239e0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.447122 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3ed10e48-0ef5-4200-8a5a-075ff239e0d8 None None] MFA Rules not processed for user `441c7f4882d74eaabbe434654e9d862d`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:54:18.448320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3ed10e48-0ef5-4200-8a5a-075ff239e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:18.459111 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4411/8891] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:54:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:54:18.475526 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:18.486771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e567cb-0fcf-4502-9323-d6c8b653a94a service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:18.492049 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:18.492649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.492928 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.493200 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.501517 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:18.503131 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:18.512702 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4481/8892] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:18.681640 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.681996 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.682281 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.702786 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:54:18.703967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:18.713640 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4412/8893] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 21:54:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:54:18.724416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:18.733706 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163be48-8b1f-41ff-95ce-ded849b6cc9d service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:18.737294 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:18.737914 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:18.738185 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:18.738450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:18.748315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:18.749720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:18.761013 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4482/8894] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:19.546442 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:19.556360 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9230ed6-384c-4350-a968-2aa2a885f941 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:19.560254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:19.560921 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:19.561285 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:19.561630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:19.563472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:19.577675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:19.579266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:19.590847 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4413/8895] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:19.733749 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:19.743314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275033a-ac15-4f02-b7b6-230b7201e2b5 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:19.746948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:19.747535 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:19.747812 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:19.748083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:19.749799 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:19.759512 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:19.760969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:19.769000 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4483/8896] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:25.448515 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:25.460534 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cd5c0-7109-483e-b99c-30c77fa2cebc service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:25.465962 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:25.466581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:25.466855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:25.467140 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:25.475120 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:25.476673 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:25.485441 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4414/8897] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:29.765605 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=36 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:54:35.481438 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=27 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:54:37.464694 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:37.473613 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97bc5d-c4e1-47fa-8916-5a2992359510 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:37.477328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:37.477925 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:37.478206 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:37.478482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:37.480263 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:37.490169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:37.491684 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:37.499771 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4484/8898] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:38.214202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:38.226284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b822618-7708-4f36-8d76-7a8d69365acf service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:38.232000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:38.232545 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:38.232818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:38.233091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:38.234880 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:38.247911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:38.249495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:38.260323 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4415/8899] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:43.376557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:54:43.392157 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b82c-6f1c-4202-8cf0-6aac013beab9 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:54:43.399724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:54:43.402594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:54:43.402594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:54:43.402594 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:54:43.403019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:54:43.417751 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:54:43.419495 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64401b-9820-4533-afff-7c4ce9ffb0d6 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:54:43.431840 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4485/8900] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:54:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:54:48.255871 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:54:53.426624 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:55:24.689443 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:55:24.698911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92795d80-2138-4782-962d-f2969b1c9f62 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:55:24.702536 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:55:24.703126 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:55:24.703402 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:55:24.703675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:55:24.713083 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:55:24.714727 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:55:24.724743 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4416/8901] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:55:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:55:27.374931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:55:27.375440 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:55:27.375545 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:55:27.397962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:55:27.399292 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:55:27.410191 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4486/8902] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:55:28.156746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:55:28.169507 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27779348-5f21-4232-bf46-7e6b1fb8cf76 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:55:28.175583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:55:28.176198 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:55:28.176472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:55:28.177192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:55:28.188535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:55:28.190136 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:55:28.204100 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4417/8903] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:55:36.583480 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:55:36.595369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845130d-1fe4-4520-a129-7dd004be6364 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:55:36.600859 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:55:36.601727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:55:36.602006 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:55:36.602277 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:55:36.610657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:55:36.612221 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b219-209a-4a8a-8287-f287a5daf73c service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:55:36.623060 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4487/8904] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:55:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:55:38.198361 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=12 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:55:46.618924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=9 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:56:45.376593 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:56:45.387310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512aed0-5ebf-4aa2-a4eb-4e4aa81f9cfb service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:56:45.392256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:56:45.392832 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:56:45.393104 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:56:45.393369 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:56:45.402310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:56:45.403907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:56:45.413364 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4418/8905] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:56:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:56:55.409421 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:57:15.769271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:57:15.769271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:57:15.769755 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:57:15.796313 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] MFA Rules not processed for user `ce56b96d91fa42029081bd0e6eccb5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:57:15.797819 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:57:15.807225 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4488/8906] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:57:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:57:16.609377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:57:16.626227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86006b-4172-4e23-af5b-eca27ba0660f service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:57:16.636961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:57:16.637561 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:57:16.639155 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:57:16.639606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:57:16.651953 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:57:16.653558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:57:16.666625 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4419/8907] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:57:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:57:25.790304 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:57:26.661332 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:40.424603 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:40.474976 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb127d8-62f7-4459-a1e7-afd10f424850 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:40.482829 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:40.483727 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:40.484004 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:40.484271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:40.501629 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:40.503247 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:40.515489 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4489/8908] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 21:58:47.197460 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:47.243466 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0b6c6-b399-4caf-8499-f633881f9f52 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:47.249411 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:47.250022 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:47.250472 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:47.251059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:47.261124 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:47.262869 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1ec9d-d446-4e9d-a33a-334760671d09 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:47.274311 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4420/8909] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:58:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:58:47.507272 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:47.590746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82b400-f59f-4e3b-a5af-b12b8001f35a service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:47.595366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:47.596010 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:47.596286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:47.596556 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] PATH_INFO: `/v3/users/ce56b96d91fa42029081bd0e6eccb5e4` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:47.597654 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce56b96d91fa42029081bd0e6eccb5e4)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:47.599601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:47.648771 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce56b96d91fa42029081bd0e6eccb5e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:47.649414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce56b96d91fa42029081bd0e6eccb5e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:47.656746 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce56b96d91fa42029081bd0e6eccb5e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:47.669576 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce56b96d91fa42029081bd0e6eccb5e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:47.670226 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce56b96d91fa42029081bd0e6eccb5e4'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:47.682546 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4490/8910] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:58:47 2022] DELETE /identity/v3/users/ce56b96d91fa42029081bd0e6eccb5e4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 21:58:47.687175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:47.687455 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:47.687726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:47.737015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:47.738232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:47.797393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4421/8911] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:58:47.807711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:47.809114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:47.841997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:47.901896 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7762c-449a-4c3a-8cc5-9916635a2183 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:47.905927 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:47.906568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:47.906851 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:47.907127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:47.918097 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:47.919779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:47.932463 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4491/8912] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:58:48.113623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:48.115091 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:48.127444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4ea9b-b5eb-424b-8277-bd257bfb20af None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:48.133067 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:48.133710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.134181 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.134459 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] PATH_INFO: `/v3/projects/22a8ea7438b641139afc6ea3c2dcba4c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.135630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a8ea7438b641139afc6ea3c2dcba4c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:48.137546 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:48.184751 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a8ea7438b641139afc6ea3c2dcba4c', 'request_id': 'req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.190354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a8ea7438b641139afc6ea3c2dcba4c', 'request_id': 'req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.197942 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a8ea7438b641139afc6ea3c2dcba4c', 'request_id': 'req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.209005 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] The token cache is being invalidate because project 22a8ea7438b641139afc6ea3c2dcba4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:58:48.209226 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.210580 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.212081 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4422/8913] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:58:48 2022] DELETE /identity/v3/projects/22a8ea7438b641139afc6ea3c2dcba4c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:48.216095 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:48.225772 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d15c3f-f070-4995-a160-092e16f41bc1 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:48.283676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:48.284466 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.284844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.284984 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] PATH_INFO: `/v3/users/3ce7e2cd8de64a86827b47417a26c7aa` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.286146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce7e2cd8de64a86827b47417a26c7aa)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:48.288918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:48.341930 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce7e2cd8de64a86827b47417a26c7aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.342712 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce7e2cd8de64a86827b47417a26c7aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.351543 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce7e2cd8de64a86827b47417a26c7aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.364347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce7e2cd8de64a86827b47417a26c7aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.365033 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-ef8f1cfb-b535-4083-bec3-fb1de0819fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce7e2cd8de64a86827b47417a26c7aa'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.378739 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4492/8914] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 21:58:48 2022] DELETE /identity/v3/users/3ce7e2cd8de64a86827b47417a26c7aa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:48.660070 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:48.661517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:48.677072 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d84f-f3d0-4283-b070-18dfb35d9f8b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:48.688834 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:48.740978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:48.742397 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.742710 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.742977 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] PATH_INFO: `/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.744092 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7fcf14adabe4ef2b3d9094cb80395cb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:48.746051 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:48.793080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7fcf14adabe4ef2b3d9094cb80395cb', 'request_id': 'req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.798781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7fcf14adabe4ef2b3d9094cb80395cb', 'request_id': 'req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.806750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7fcf14adabe4ef2b3d9094cb80395cb', 'request_id': 'req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.818196 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] The token cache is being invalidate because project f7fcf14adabe4ef2b3d9094cb80395cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 21:58:48.818484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.819684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-ab02a99a-8b9a-4e16-ae69-24c84b3eacc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 21:58:48.821159 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4423/8915] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 21:58:48 2022] DELETE /identity/v3/projects/f7fcf14adabe4ef2b3d9094cb80395cb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:48.827068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.827567 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.827666 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.855268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:48.879104 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:48.880310 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:48.938727 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4493/8916] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:58:48.947493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.947778 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.948061 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.967813 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:48.969009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:48.977058 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4424/8917] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:58:48.981407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:48.991353 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52af492-34cf-40ce-a5bf-cf01c6662b65 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:48.995996 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:48.996589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:48.996872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:48.997146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:48.997785 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 21:58:48.998091 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:48.999732 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.011678 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4494/8918] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 21:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 21:58:49.015918 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.025750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a329c6-4851-4b9a-ab8f-a7926990eb5c None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.030376 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a588a-1159-4555-a149-22de7189420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.030997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a588a-1159-4555-a149-22de7189420b admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.031279 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a588a-1159-4555-a149-22de7189420b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.031556 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a588a-1159-4555-a149-22de7189420b admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.032185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a588a-1159-4555-a149-22de7189420b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.034082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a588a-1159-4555-a149-22de7189420b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.070722 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4425/8919] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:58:49 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:58:49.074905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.085665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fb90-ec25-4b94-8fad-3a69e7abeaa7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.145423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.146011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.146286 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.146584 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.147924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.150067 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.170866 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4495/8920] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:58:49 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:58:49.175109 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a588a-1159-4555-a149-22de7189420b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.186595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a588a-1159-4555-a149-22de7189420b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.191160 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.191748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.192027 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.192295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.192923 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.195532 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.205351 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4426/8921] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:58:49.209507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.220114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fca038-9dd0-420b-85e9-73b5f407c7d4 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.224768 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.225449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.225932 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.226291 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.226979 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.228546 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134cdb-047d-4d22-99dd-d159bc1e706d admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.239287 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4496/8922] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:58:49.244082 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.253886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b553b0-009d-4376-8a83-9bba32af03fe admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.259490 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.260125 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.260408 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.260682 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] PATH_INFO: `/v3/projects/8a9daa5e226649d08e09774215b0b7d0/users/0aaf38856de9425ab9909937c85b151c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.275013 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9daa5e226649d08e09774215b0b7d0, user_id=0aaf38856de9425ab9909937c85b151c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.277654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.290360 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4427/8923] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:58:49 2022] PUT /identity/v3/projects/8a9daa5e226649d08e09774215b0b7d0/users/0aaf38856de9425ab9909937c85b151c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:49.295568 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412d9b2-35ea-4515-bcd7-66e9ce22d257 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.295855 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412d9b2-35ea-4515-bcd7-66e9ce22d257 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.296161 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412d9b2-35ea-4515-bcd7-66e9ce22d257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.315997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-5412d9b2-35ea-4515-bcd7-66e9ce22d257 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:49.317230 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5412d9b2-35ea-4515-bcd7-66e9ce22d257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:49.377173 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4497/8924] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:58:49.387985 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:49.389660 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:49.423254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:49.449392 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb708a2d-6d68-4dbd-b1b4-6adadededd6e admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:49.453075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:49.453675 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:49.453954 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:49.454230 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:49.463320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:49.464753 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:49.474160 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4428/8925] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:58:53.582255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:53.582844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:53.582844 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:53.590764 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:53.614972 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] MFA Rules not processed for user `0aaf38856de9425ab9909937c85b151c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:53.616327 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:53.686755 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4498/8926] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:58:53.691440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:53.704007 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37e326-01a5-478e-b38b-55c3caf240b8 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:53.709590 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:53.710232 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:53.710610 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:53.710949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] PATH_INFO: `/v3/projects` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:53.711654 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:53.713963 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:53.753492 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4429/8927] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 21:58:53 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:58:53.762242 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:53.773412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01583-362f-4d6a-9694-2c9201e3ea0a None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:53.847639 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:53.848456 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:53.848729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:53.848837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:53.849506 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:53.851642 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:53.874878 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4499/8928] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 21:58:53 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 21:58:53.878961 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:53.893372 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8024f9-90b9-4f25-bdd6-80182c7a2086 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:53.898327 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:53.898987 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:53.899266 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:53.899535 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:53.900375 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:53.902329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:53.915411 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4430/8929] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 21:58:53.919623 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:53.931806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeae79-6ad2-495b-97d6-f46f60007ac1 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:53.937686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:53.939088 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:53.939426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:53.939636 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] PATH_INFO: `/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07/users/090b44706c414751b70e3c1fdc21563b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:53.957969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8015e5dc3f4f1a9b9760457cbeac07, user_id=090b44706c414751b70e3c1fdc21563b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:53.961068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:53.975124 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4500/8930] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:58:53 2022] PUT /identity/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07/users/090b44706c414751b70e3c1fdc21563b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:53.979427 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:53.991855 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e1e3-ed09-4a84-b4d3-5b4862b7267b admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:54.051284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:54.051895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:54.052306 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:54.052592 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:54.053896 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:54.055557 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf247e5-6b30-49c9-a82e-0ab52f52e840 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:54.070581 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4431/8931] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 21:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 21:58:54.072019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:54.082939 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e61da-7cf1-4845-b98e-aad3cb388db5 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:54.087876 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:54.088539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:54.088820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:54.089112 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] PATH_INFO: `/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07/users/090b44706c414751b70e3c1fdc21563b/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:54.091193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8015e5dc3f4f1a9b9760457cbeac07, user_id=090b44706c414751b70e3c1fdc21563b, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:54.093846 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:54.106230 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4501/8932] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 21:58:54 2022] PUT /identity/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07/users/090b44706c414751b70e3c1fdc21563b/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 21:58:58.327322 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb70b4-8e19-4619-8f3c-bcf4303c8363 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:58.327882 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb70b4-8e19-4619-8f3c-bcf4303c8363 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:58.328075 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb70b4-8e19-4619-8f3c-bcf4303c8363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:58.397271 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-e9cb70b4-8e19-4619-8f3c-bcf4303c8363 None None] MFA Rules not processed for user `090b44706c414751b70e3c1fdc21563b`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:58.399012 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e9cb70b4-8e19-4619-8f3c-bcf4303c8363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:58.557121 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4432/8933] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 21:58:58.577063 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:58:58.580058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:58.689452 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:58:58.734019 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef64fa6-999b-4aae-b7b1-cbd4d01bf0a0 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:58:58.741837 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:58:58.742479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:58.742801 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:58.743078 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:58.820371 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:58:58.822315 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:58:58.833524 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4502/8934] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:58:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:58:59.935235 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:58:59.936074 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:58:59.936353 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:58:59.955848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] MFA Rules not processed for user `0aaf38856de9425ab9909937c85b151c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 21:58:59.957058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:58:59.964045 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4433/8935] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 21:58:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 21:59:00.305788 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:59:00.362511 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b2da9-4e4b-41c4-b9f8-293ceda9e152 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:59:00.368246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:59:00.368884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:59:00.369166 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:59:00.369439 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:59:00.371207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:59:00.381657 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:59:00.383154 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:59:00.391158 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4503/8936] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:59:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:59:09.950265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:59:10.387640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=62 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:59:13.165040 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:59:13.174524 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df3f2d-cbde-4e1f-a464-cec7b4bc49e2 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:59:13.178950 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:59:13.179539 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:59:13.179816 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:59:13.180094 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:59:13.188866 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:59:13.190288 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:59:13.200346 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4434/8937] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:59:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:59:23.196202 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 21:59:44.015792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 21:59:44.024820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad086f5c-cfb8-442d-9315-3bcd4f513bea service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 21:59:44.028476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 21:59:44.029092 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 21:59:44.029372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 21:59:44.029646 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 21:59:44.031424 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 21:59:44.043977 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 21:59:44.045410 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 21:59:44.053758 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4504/8938] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 21:59:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 21:59:54.049944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:00:40.626607 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:00:40.628000 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:00:40.718604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2df5d-4179-4580-941b-29235c82bc54 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:00:40.723659 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:00:40.724280 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:00:40.724558 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:00:40.724828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:00:40.726573 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:00:40.738606 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:00:40.740184 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:00:40.750104 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4435/8939] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:00:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:00:50.746018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=41 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:10.341669 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:10.343270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:10.421451 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03abfd7-bf63-418b-a885-bad1ba93423b service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:10.425311 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:10.425901 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:10.426176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:10.426444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:10.428176 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:10.442369 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:10.443845 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527214e-e026-4a1f-9907-f623393f20de service glance] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:10.455224 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4505/8940] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 22:01:10.643984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:10.654192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd13076-e4ca-4231-805a-73f82d668871 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:10.657967 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:10.658597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:10.658878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:10.659146 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:10.660851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:10.674522 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:10.676222 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:10.686883 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4436/8941] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:20.450455 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=35 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:20.682148 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:23.457444 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:23.457955 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:23.458099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:23.521769 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:01:23.523223 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:23.593164 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4506/8942] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 22:01:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:01:23.667563 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:23.669108 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:23.777520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43168326-add2-4f7e-87d5-143e4fc0c281 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:23.791293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:23.791917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:23.792195 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:23.792878 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:23.801684 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:23.803906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:23.812314 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4437/8943] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:01:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:01:25.348068 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:25.374924 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0383e7-a721-482e-a283-88d28e5fa5bf None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:25.381137 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:25.382328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:25.382990 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:25.383413 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:25.385269 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:25.403386 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:25.405437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eb122-1f98-4390-82e1-7820ead531c3 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:25.418709 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4507/8944] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:33.809465 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=41 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:35.413779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=50 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:36.177374 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:36.187037 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2899a-7740-4e14-8a34-5448048d534a service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:36.190555 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:36.191159 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:36.191430 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:36.191697 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:36.194058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:36.203096 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:36.204501 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:36.212767 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4438/8945] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:37.340343 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:37.341407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:37.342794 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:37.403739 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] MFA Rules not processed for user `6531dafbedef45ccb7b6b7e2f610c2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:01:37.404974 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:37.421851 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4508/8946] 10.222.0.52 () {58 vars in 1018 bytes} [Sat Sep 24 22:01:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:01:37.434227 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:37.444089 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999692-38e6-4d00-9a77-0092e39f1f80 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:37.447729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:37.448326 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:37.448611 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:37.448885 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:37.459591 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:37.461039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:37.472052 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4439/8947] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:37.556289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:37.565821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b100cee-ec92-42b8-b914-46e791972d9b None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:37.569649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:37.570241 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:37.570561 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:37.570839 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:37.572572 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:37.582228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:37.583690 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:37.592088 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4509/8948] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:37.666139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:37.676058 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d598a29-a97f-4686-8ec4-da774d57c78c service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:37.679856 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:37.680440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:37.680712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:37.680978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:37.682703 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:37.696484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:37.697897 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:37.708510 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4440/8949] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:38.591719 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:38.601053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea521cc5-9aef-4aac-8c9c-dd31b1a4fd78 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:38.604781 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:38.605379 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:38.605658 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:38.605959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:38.607724 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:38.618193 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:38.619671 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15869862-75ab-4afc-9c3c-0e8b4fdfc050 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:38.627917 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4510/8950] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:39.488865 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:39.490958 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:39.506175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ac46e-2e93-4021-8b96-24c8fa8f7803 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:39.512310 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:39.512911 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:39.513192 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:39.513470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:39.515470 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:39.529581 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:39.532559 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:39.544764 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4441/8951] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:40.210520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:40.210994 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:40.211115 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:40.231152 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:01:40.232368 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:40.242018 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4511/8952] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 22:01:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:01:40.254776 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:40.266254 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50a50-dbd4-4781-ad15-70ef5ce3602d service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:40.271595 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:40.272189 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:40.272630 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:40.272966 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:40.281028 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:40.282645 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e775f-7bab-4909-a6c6-a94f579058b6 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:40.291191 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4442/8953] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:01:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:01:50.225982 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=24 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:50.287957 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=30 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:01:51.284620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:51.297083 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cda184-2e2c-4eb8-8c8f-69013068e8ed None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:51.302219 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:51.302862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:51.303140 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:51.303407 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:51.305106 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:51.316262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:51.317821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:51.327973 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4512/8954] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:01:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:01:57.652488 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c356c-87db-4f04-b1a8-b4287a9319b8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:57.652906 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c356c-87db-4f04-b1a8-b4287a9319b8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:57.653042 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c356c-87db-4f04-b1a8-b4287a9319b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:57.672746 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-b97c356c-87db-4f04-b1a8-b4287a9319b8 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:01:57.673952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-b97c356c-87db-4f04-b1a8-b4287a9319b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:01:57.681393 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4443/8955] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 22:01:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:01:57.693704 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:01:57.704873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e7679-64d5-4d28-a20b-5e6c2526a8aa service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:01:57.710520 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:01:57.711127 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:01:57.711405 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:01:57.711674 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:01:57.720270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:01:57.721848 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:01:57.730845 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4513/8956] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:01:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:02:07.667724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:02:07.727397 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:03:49.356574 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:49.357080 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:49.357187 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:49.376441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] MFA Rules not processed for user `0aaf38856de9425ab9909937c85b151c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:03:49.377634 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:49.384491 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4444/8957] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:03:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:03:51.216309 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:51.225731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efe14-c132-43f8-86d4-9343a4c868e0 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:51.229399 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:51.229988 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:51.230270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:51.230649 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:51.291675 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:51.293813 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:51.313291 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4514/8958] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:03:54.490984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:54.493748 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:54.510952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946b3fe-78d2-4820-a627-71e3bbfd4d25 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:54.516874 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:54.517480 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:54.517761 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:54.518032 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:54.519840 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:54.531735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:54.533328 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:54.545333 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4445/8959] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:03:57.474164 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:57.486005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a38fe-a6c1-4e24-9c68-58dc779836a0 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:57.491233 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:57.491908 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:57.492259 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:57.492601 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:57.494566 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:57.507475 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:57.509482 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:57.521561 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4515/8960] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:03:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:03:57.537825 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:57.550870 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17227d2-fa0e-415b-89e3-9a5014aed436 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:57.556893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:57.557635 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:57.558131 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:57.558363 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] PATH_INFO: `/v3/users/0aaf38856de9425ab9909937c85b151c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:57.561167 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aaf38856de9425ab9909937c85b151c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:57.563224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:57.613978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aaf38856de9425ab9909937c85b151c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.614681 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aaf38856de9425ab9909937c85b151c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.622620 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aaf38856de9425ab9909937c85b151c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.629696 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aaf38856de9425ab9909937c85b151c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.630282 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aaf38856de9425ab9909937c85b151c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.652175 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4446/8961] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 22:03:57 2022] DELETE /identity/v3/users/0aaf38856de9425ab9909937c85b151c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:03:57.838271 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:57.849650 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f81b-667b-406d-8dc1-0c2690ef1862 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:57.910754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:57.911489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:57.911872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:57.912220 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] PATH_INFO: `/v3/projects/8a9daa5e226649d08e09774215b0b7d0` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:57.913426 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9daa5e226649d08e09774215b0b7d0)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:57.915412 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:57.966777 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9daa5e226649d08e09774215b0b7d0', 'request_id': 'req-444ca0cd-8a3e-4182-888b-0c1472cff209'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.972449 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9daa5e226649d08e09774215b0b7d0', 'request_id': 'req-444ca0cd-8a3e-4182-888b-0c1472cff209'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.979225 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9daa5e226649d08e09774215b0b7d0', 'request_id': 'req-444ca0cd-8a3e-4182-888b-0c1472cff209'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.992847 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] The token cache is being invalidate because project 8a9daa5e226649d08e09774215b0b7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 22:03:57.993136 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.993762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:57.995306 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4516/8962] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 22:03:57 2022] DELETE /identity/v3/projects/8a9daa5e226649d08e09774215b0b7d0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:03:58.000211 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.001680 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:58.016842 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d049a-544d-48f2-a1d3-243aa8edc6e6 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.072493 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.073110 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.073379 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.073685 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] PATH_INFO: `/v3/users/090b44706c414751b70e3c1fdc21563b` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.075844 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090b44706c414751b70e3c1fdc21563b)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.077757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.124106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090b44706c414751b70e3c1fdc21563b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.124828 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090b44706c414751b70e3c1fdc21563b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.132383 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090b44706c414751b70e3c1fdc21563b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.133609 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:03:58.142296 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090b44706c414751b70e3c1fdc21563b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.143324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-5f32e6f0-9b28-4091-bb10-ee5cddd79bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090b44706c414751b70e3c1fdc21563b'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.160227 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4447/8963] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 22:03:57 2022] DELETE /identity/v3/users/090b44706c414751b70e3c1fdc21563b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:03:58.334038 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.335640 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:58.343922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:03:58.349711 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ca0cd-8a3e-4182-888b-0c1472cff209 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.408742 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.409372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.409663 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.409917 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] PATH_INFO: `/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.411046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8015e5dc3f4f1a9b9760457cbeac07)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.412937 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.458943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8015e5dc3f4f1a9b9760457cbeac07', 'request_id': 'req-66f53eaa-156f-4a38-9dd7-720866680a61'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.464329 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8015e5dc3f4f1a9b9760457cbeac07', 'request_id': 'req-66f53eaa-156f-4a38-9dd7-720866680a61'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.470378 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8015e5dc3f4f1a9b9760457cbeac07', 'request_id': 'req-66f53eaa-156f-4a38-9dd7-720866680a61'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.483382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] The token cache is being invalidate because project cd8015e5dc3f4f1a9b9760457cbeac07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 22:03:58.483993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.484268 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-66f53eaa-156f-4a38-9dd7-720866680a61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:03:58.485732 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4517/8964] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 22:03:58 2022] DELETE /identity/v3/projects/cd8015e5dc3f4f1a9b9760457cbeac07 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:03:58.491031 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.491320 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.491597 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.512639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:03:58.513827 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:58.573733 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4448/8965] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:03:58.581585 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.581872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.582150 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.602731 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:03:58.603943 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:58.612769 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4518/8966] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:03:58.617015 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.627381 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc25c4b-d03d-4240-a031-bdc771f595ab None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.632147 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.632743 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.633018 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.633284 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.633930 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=99133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 22:03:58.634225 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.635984 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.648678 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4449/8967] 10.222.0.52 () {58 vars in 1153 bytes} [Sat Sep 24 22:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 22:03:58.653776 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.663190 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592d07-bb8b-4664-8a2e-5f67dc0a9060 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.667918 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.668519 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.668898 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.669177 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.669833 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.671779 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.700106 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4519/8968] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 22:03:58 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 22:03:58.704396 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.714259 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3137df3-453a-4865-9adf-4eacdbfbf3ce admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.771757 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.772350 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.772631 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.772917 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.773516 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.775461 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.796530 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4450/8969] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 22:03:58 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 22:03:58.800810 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.812387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1272d-39c8-4f17-973a-1909333e6ba2 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.817005 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.817587 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.817860 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.818126 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.818782 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.820382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.831049 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4520/8970] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 22:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 22:03:58.834900 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.845814 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d73a-6e8f-4866-8619-112b5d77cd20 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.850766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.851444 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.851724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.852021 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] PATH_INFO: `/v3/roles` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.852653 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.854307 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415ba0d-7023-4aaf-8d2b-f7c018537471 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.865016 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4451/8971] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 22:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 22:03:58.869216 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:58.880537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366d5ed-de84-4be6-acd7-606252324af7 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:58.885172 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:58.885806 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] REQUEST_METHOD: `PUT` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.886082 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.886352 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] PATH_INFO: `/v3/projects/55b9491baa5346a3a32a3d7d221b88e9/users/ace8f69828e64b3b826cc57835dfd04c/roles/ca0ddc6d9bf24f56b033bf1e448069fb` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.901372 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55b9491baa5346a3a32a3d7d221b88e9, user_id=ace8f69828e64b3b826cc57835dfd04c, role_id=ca0ddc6d9bf24f56b033bf1e448069fb)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:58.904262 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:58.917448 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4521/8972] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 22:03:58 2022] PUT /identity/v3/projects/55b9491baa5346a3a32a3d7d221b88e9/users/ace8f69828e64b3b826cc57835dfd04c/roles/ca0ddc6d9bf24f56b033bf1e448069fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:03:58.922663 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286163d-09f4-4afb-9d38-0775c4752837 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:58.922952 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286163d-09f4-4afb-9d38-0775c4752837 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:58.923224 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286163d-09f4-4afb-9d38-0775c4752837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:58.942676 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-4286163d-09f4-4afb-9d38-0775c4752837 None None] MFA Rules not processed for user `b34a84022bc34fe0bb926f7c649f3ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:03:58.943905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-4286163d-09f4-4afb-9d38-0775c4752837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:58.976293 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:03:59.006020 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4452/8973] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:03:59.016326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:03:59.018050 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:03:59.050416 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:03:59.077670 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751172-4b4f-4da3-928a-8faf07040147 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:03:59.081382 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:03:59.081922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:03:59.082201 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:03:59.082582 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:03:59.091805 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:03:59.093228 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:03:59.102704 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4522/8974] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:04:03.139278 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.139853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.139853 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.170324 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] MFA Rules not processed for user `ace8f69828e64b3b826cc57835dfd04c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:04:03.171571 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:03.241425 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4453/8975] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:04:03.246204 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.256387 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545bdc4-466f-4087-9eb2-b269e11f06b3 service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.261113 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.261720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.262207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.262317 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] PATH_INFO: `/v3/projects` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.262959 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.264869 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.294542 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4523/8976] 10.222.0.52 () {60 vars in 1153 bytes} [Sat Sep 24 22:04:03 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 22:04:03.298173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.311137 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b268f0-623c-4e8b-817b-1806512420fd None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.370139 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.370862 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.371143 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.371970 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] PATH_INFO: `/v3/users` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.372623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.374658 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.398856 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4454/8977] 10.222.0.52 () {60 vars in 1144 bytes} [Sat Sep 24 22:04:03 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 22:04:03.403109 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.414099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfaf24-10a3-407a-94b3-9788c49cfccd admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.418873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.419479 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.419760 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.420029 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.420665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.422258 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.434817 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4524/8978] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 22:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 22:04:03.439239 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.450273 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73848b6-bc9e-4c96-92ac-dcc5146aba60 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.455038 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.455695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.455971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.456241 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] PATH_INFO: `/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744/users/83bfcc7ad6e24ce18e5e2c44f26757f7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.472694 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b59236ce4de4fd0b1d79a1a32b79744, user_id=83bfcc7ad6e24ce18e5e2c44f26757f7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.475399 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.490902 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4455/8979] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 22:04:03 2022] PUT /identity/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744/users/83bfcc7ad6e24ce18e5e2c44f26757f7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 22:04:03.494886 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.506180 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5904f2d-4e20-4743-9b9c-d452555d083a admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.566968 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.567596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.567874 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.568144 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.568778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.570391 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b94ede-e94d-47cc-9a9f-61a138fd6b90 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.581501 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4525/8980] 10.222.0.52 () {58 vars in 1122 bytes} [Sat Sep 24 22:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 22:04:03.585780 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:03.597495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6823ad-ee01-4b71-a848-10d440cff7bb admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:03.602261 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:03.602939 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] REQUEST_METHOD: `PUT` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:03.603220 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:03.603497 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] PATH_INFO: `/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744/users/83bfcc7ad6e24ce18e5e2c44f26757f7/roles/49b89b74bd2a42b6ba0c6496570ab8dc` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:03.605735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b59236ce4de4fd0b1d79a1a32b79744, user_id=83bfcc7ad6e24ce18e5e2c44f26757f7, role_id=49b89b74bd2a42b6ba0c6496570ab8dc)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:03.608712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:03.620474 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4456/8981] 10.222.0.52 () {60 vars in 1483 bytes} [Sat Sep 24 22:04:03 2022] PUT /identity/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744/users/83bfcc7ad6e24ce18e5e2c44f26757f7/roles/49b89b74bd2a42b6ba0c6496570ab8dc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:04:08.154718 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8fdf3-852d-490e-9926-081e3bbd4256 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:08.155645 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8fdf3-852d-490e-9926-081e3bbd4256 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:08.156102 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8fdf3-852d-490e-9926-081e3bbd4256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:08.208086 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-94e8fdf3-852d-490e-9926-081e3bbd4256 None None] MFA Rules not processed for user `83bfcc7ad6e24ce18e5e2c44f26757f7`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:04:08.209359 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-94e8fdf3-852d-490e-9926-081e3bbd4256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:08.333314 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4526/8982] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:04:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:04:08.351639 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:08.354908 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:08.393768 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:04:08.480705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efa27e-5722-43c9-b92b-01df55d8da43 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:08.488941 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:08.489978 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:08.490354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:08.490756 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:08.607252 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:08.609014 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:08.620243 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4457/8983] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:04:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 22:04:09.685870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:09.686390 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:09.686539 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:09.705754 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] MFA Rules not processed for user `ace8f69828e64b3b826cc57835dfd04c`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:04:09.706992 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:09.713841 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4527/8984] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:04:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:04:10.092151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:10.094784 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:10.168705 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c6fa2-3332-47cb-9115-1428ff294440 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:10.174336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:10.174991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:10.175272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:10.175569 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:10.177354 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:10.189178 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:10.190752 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:10.200277 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4458/8985] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:04:10.569816 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:10.571255 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:10.586053 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:04:10.659500 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59408ed2-ac7f-4922-9574-f4c5399008cd None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:10.664514 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:10.665124 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:10.665406 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:10.665682 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:10.667508 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:10.678970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:10.680476 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:10.691138 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4528/8986] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:04:20.196203 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=73 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:04:20.687020 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=26 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:04:22.855079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:22.864578 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663cdd7-1601-4cec-b8e6-512b9e97a39a service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:22.868735 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:22.869321 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:22.869598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:22.869938 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:22.879378 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:22.880799 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:22.890872 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4459/8987] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:04:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:04:32.886517 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:04:47.026295 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:04:47.037326 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc405608-001e-46d5-bc6b-b1bc5b27b246 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:04:47.042236 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:04:47.042866 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:04:47.043146 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:04:47.043446 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:04:47.045557 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:04:47.055541 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:04:47.056983 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c019f2-15b6-4675-a123-4ad2de491784 service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:04:47.066095 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4529/8988] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:04:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:04:57.062046 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:05:47.619236 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:05:47.620622 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:05:47.636016 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8f0d8-19c7-4ce3-b56a-75def59ea6f6 service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:05:47.640713 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:05:47.642157 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:05:47.642644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:05:47.642922 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:05:47.644602 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:05:47.656111 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:05:47.658689 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:05:47.668219 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4460/8989] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:05:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:05:57.664175 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:06:19.915913 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05732-68d7-43d1-9344-9f02613ce9b2 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:19.916362 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05732-68d7-43d1-9344-9f02613ce9b2 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:19.916464 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e05732-68d7-43d1-9344-9f02613ce9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:19.954878 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-80e05732-68d7-43d1-9344-9f02613ce9b2 None None] MFA Rules not processed for user `99f0d8a1aa0d4bacbd4e4fb2e7864cee`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:06:19.956089 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-80e05732-68d7-43d1-9344-9f02613ce9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:20.006263 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4530/8990] 10.222.0.52 () {58 vars in 1059 bytes} [Sat Sep 24 22:06:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:06:20.012441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:20.022185 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06726-7905-418a-85ca-2835f1e0707a service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:20.025803 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f0d8a1aa0d4bacbd4e4fb2e7864cee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:20.026384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:20.026695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:20.026969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:20.028647 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:20.061495 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:20.063118 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:20.083885 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4461/8991] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 22:06:24.189814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:24.190289 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:24.190414 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:24.210686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:06:24.211870 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:24.266972 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4531/8992] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 22:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:06:24.312520 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:24.313919 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:24.387041 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc4d63-762f-41cb-9543-5de4332d1884 service glance] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:24.392009 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:24.392613 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:24.392895 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:24.393168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:24.401272 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:24.402892 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d720f37-8be9-490f-9fad-5de0616186b0 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:24.411180 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4462/8993] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:06:26.996085 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:27.016818 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a85f8-40d4-40c7-b1cc-5706d1dc5a4d None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:27.024235 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:27.024862 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:27.025147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:27.025423 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:27.028114 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:27.041011 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:27.042798 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:27.054118 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4532/8994] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 22:06:33.773758 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:33.774286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:33.774286 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:33.793484 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] MFA Rules not processed for user `ace8f69828e64b3b826cc57835dfd04c`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:06:33.795035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:33.801997 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4463/8995] 10.222.0.52 () {58 vars in 958 bytes} [Sat Sep 24 22:06:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:06:34.438922 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:34.442907 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:34.451195 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd50d1-3e11-4ace-9336-f2b5b81aeb27 service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:34.452463 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6444f-57a4-4444-a344-84a07fbcaae8 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:34.455991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:34.456339 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:34.456712 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:34.456948 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:34.457059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:34.457256 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:34.457366 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:34.457602 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:34.459036 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:34.459305 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:34.471328 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:34.473617 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d12283-7fc1-4c55-a522-15997595d916 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:34.476833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:34.478263 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:34.483722 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4533/8996] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:06:34.490427 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4464/8997] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 22:06:36.748881 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:36.749370 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:36.749489 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:36.772528 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:06:36.773821 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:36.781338 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4534/8998] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 22:06:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:06:36.794990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:36.817728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030b2f-30cc-4156-98b7-5df5a6962d9f service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:36.824791 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:36.825623 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:36.825901 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:36.826168 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:36.836200 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:36.838336 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:36.855702 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4465/8999] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:06:46.766434 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=69 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:06:46.843949 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=65 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:06:46.892715 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:46.903721 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093514-d26c-46b9-9fc8-31322c4e21c7 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:46.907723 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:46.908319 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:46.908620 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:46.908888 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:46.911377 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:46.920873 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:46.922314 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:46.930536 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4535/9000] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:06:47.668893 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:47.670528 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:47.689633 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a0b2-3e8c-4d38-b6af-19a5e21ed19b service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:47.696384 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:47.697039 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:47.697312 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:47.697583 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:47.699316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:47.712020 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:47.713603 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:47.724429 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4466/9001] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:06:56.300790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:06:56.312946 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e726f4a-59cb-41fc-8ce4-0c3712237a4c service neutron] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:06:56.318884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:06:56.319716 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:06:56.319997 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:06:56.320270 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:06:56.322003 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:06:56.334614 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:06:56.336192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461f5e5-282d-4d64-8c58-c1d35ffd4950 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:06:56.346196 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4536/9002] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:06:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:06:57.719766 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:07:06.341905 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=12 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:11.615464 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:11.658525 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4bbe1-13db-44a2-a714-0a450c007f57 service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:11.664848 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:11.665445 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:11.665726 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:11.666006 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:11.673636 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:11.675265 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:11.684378 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4467/9003] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:09:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:09:19.080695 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7d4-d11a-4455-9a83-099f62e0c7a1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:19.081175 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7d4-d11a-4455-9a83-099f62e0c7a1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:19.081282 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7d4-d11a-4455-9a83-099f62e0c7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:19.111809 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-b225e7d4-d11a-4455-9a83-099f62e0c7a1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:09:19.113008 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-b225e7d4-d11a-4455-9a83-099f62e0c7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:19.130434 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4537/9004] 10.222.0.52 () {58 vars in 1009 bytes} [Sat Sep 24 22:09:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:09:19.170860 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:19.182794 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b32a42-0f59-4f09-b28d-ae94a2c0339a service nova] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:19.187745 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:19.188366 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:19.188644 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:19.188914 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:19.197992 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:19.199624 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:19.207621 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4468/9005] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:09:29.122820 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=11 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:29.204351 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=26 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:29.425953 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] REQUEST_METHOD: `POST` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:29.426246 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:29.426552 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:29.447057 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.auth.core [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:09:29.448266 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:29.455333 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4538/9006] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 22:09:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:09:29.468176 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:29.478851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ef6fb-6948-4f92-958c-d5da979a3cb9 service swift] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:29.483858 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:29.484441 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:29.484722 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:29.484991 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:29.493483 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:29.495071 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4b530-8777-4bc2-a109-ada26340d47c service swift] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:29.503005 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4469/9007] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:09:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:09:30.660147 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:30.683491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85773b3f-6194-4d90-a6e8-0f0f507e9fb1 None None] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:30.690491 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6531dafbedef45ccb7b6b7e2f610c2c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:30.692099 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:30.692477 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:30.692872 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:30.694792 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:30.710544 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:30.712852 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:30.741108 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4539/9008] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:09:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:09:34.514255 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] REQUEST_METHOD: `POST` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:34.514771 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:34.514883 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:34.534833 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.auth.core [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] MFA Rules not processed for user `f329c05a7437450db2315117edbb65ea`. Rule list: `[]` (Enabled: `True`). {{(pid=99133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 22:09:34.536062 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:34.543749 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4470/9009] 10.222.0.52 () {58 vars in 967 bytes} [Sat Sep 24 22:09:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 22:09:34.555683 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:34.567507 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea8e0-bf70-46be-a438-24542d859f6f service nova] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:34.573079 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b5a7880d6ec47f381af9ef1f09a767c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:34.573656 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:34.573931 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:34.574202 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:34.582072 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:34.583993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:34.593450 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4540/9010] 10.222.0.52 () {60 vars in 1384 bytes} [Sat Sep 24 22:09:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 22:09:38.865905 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:38.885329 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930d2c1-9323-4d17-a67e-dd89574ece25 None None] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:38.888851 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b98deaef6c24ed2aa7c7ecc90c75205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:38.889446 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:38.889728 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:38.889997 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:38.959150 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:38.960577 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:38.970914 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4471/9011] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:09:42.088903 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:42.100688 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8cfcf6-55a4-4212-8f2e-2dbd01a67690 service swift] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:42.106596 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:42.108056 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:42.110575 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:42.110856 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:42.112778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:42.125961 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:42.127676 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:42.139214 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4541/9012] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:09:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:09:45.062702 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:45.075969 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9ddf9-1eba-4fa9-806c-ea8a971501ed service neutron] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:45.080586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:45.081204 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:45.081479 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:45.081750 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] PATH_INFO: `/v3/users/ace8f69828e64b3b826cc57835dfd04c` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:45.083079 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace8f69828e64b3b826cc57835dfd04c)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:45.084990 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:45.136775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace8f69828e64b3b826cc57835dfd04c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.137440 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace8f69828e64b3b826cc57835dfd04c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.146173 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace8f69828e64b3b826cc57835dfd04c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.155481 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace8f69828e64b3b826cc57835dfd04c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.156377 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace8f69828e64b3b826cc57835dfd04c'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.175234 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4472/9013] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 22:09:45 2022] DELETE /identity/v3/users/ace8f69828e64b3b826cc57835dfd04c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:09:45.355211 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:45.366589 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc9fcf-0c16-4c3b-90fc-b852c86c5938 service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:45.426547 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:45.427207 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:45.427487 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:45.427762 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] PATH_INFO: `/v3/projects/55b9491baa5346a3a32a3d7d221b88e9` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:45.428879 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55b9491baa5346a3a32a3d7d221b88e9)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:45.431427 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:45.476863 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55b9491baa5346a3a32a3d7d221b88e9', 'request_id': 'req-c9cb2983-3443-4eec-9064-985aa085f5c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.482729 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55b9491baa5346a3a32a3d7d221b88e9', 'request_id': 'req-c9cb2983-3443-4eec-9064-985aa085f5c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.489790 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55b9491baa5346a3a32a3d7d221b88e9', 'request_id': 'req-c9cb2983-3443-4eec-9064-985aa085f5c9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.515778 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] The token cache is being invalidate because project 55b9491baa5346a3a32a3d7d221b88e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 22:09:45.516071 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.516705 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.518234 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4542/9014] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 22:09:45 2022] DELETE /identity/v3/projects/55b9491baa5346a3a32a3d7d221b88e9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:09:45.522295 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:45.523717 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:45.537598 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43587739-229e-4561-bd98-578a855995a3 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:45.563729 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:45.596156 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:45.596781 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:45.597060 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:45.597330 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] PATH_INFO: `/v3/users/83bfcc7ad6e24ce18e5e2c44f26757f7` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:45.598456 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bfcc7ad6e24ce18e5e2c44f26757f7)` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:45.600426 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:45.646662 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bfcc7ad6e24ce18e5e2c44f26757f7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.647309 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bfcc7ad6e24ce18e5e2c44f26757f7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.654779 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bfcc7ad6e24ce18e5e2c44f26757f7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.661931 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bfcc7ad6e24ce18e5e2c44f26757f7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.662586 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.notifications [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bfcc7ad6e24ce18e5e2c44f26757f7'} {{(pid=99133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.680187 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4473/9015] 10.222.0.52 () {58 vars in 1224 bytes} [Sat Sep 24 22:09:45 2022] DELETE /identity/v3/users/83bfcc7ad6e24ce18e5e2c44f26757f7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:09:45.860951 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:09:45.862419 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:09:45.877944 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb2983-3443-4eec-9064-985aa085f5c9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:09:45.910672 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:45.939720 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b34a84022bc34fe0bb926f7c649f3ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a8f45f34e55f4f50b20d34824a4e60af', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:09:45.940345 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:09:45.940622 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:09:45.940891 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] PATH_INFO: `/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:09:45.942245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b59236ce4de4fd0b1d79a1a32b79744)` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:09:45.944169 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:09:45.990347 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b59236ce4de4fd0b1d79a1a32b79744', 'request_id': 'req-5847ab52-8592-4355-99e5-2c6ef68496a9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:45.995970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b59236ce4de4fd0b1d79a1a32b79744', 'request_id': 'req-5847ab52-8592-4355-99e5-2c6ef68496a9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:46.002218 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b59236ce4de4fd0b1d79a1a32b79744', 'request_id': 'req-5847ab52-8592-4355-99e5-2c6ef68496a9'} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:46.014969 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] The token cache is being invalidate because project 1b59236ce4de4fd0b1d79a1a32b79744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=99132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 22:09:46.015192 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:46.015814 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.notifications [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=99132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 22:09:46.017307 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4543/9016] 10.222.0.52 () {58 vars in 1233 bytes} [Sat Sep 24 22:09:45 2022] DELETE /identity/v3/projects/1b59236ce4de4fd0b1d79a1a32b79744 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 22:09:55.676981 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:09:56.011884 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:10:54.625604 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:10:54.627035 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:10:54.720106 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f205e1-afc7-450a-a59d-ea025dbbac37 admin admin] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:10:54.778151 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:10:54.778775 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:10:54.779059 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:10:54.779971 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:10:54.781886 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.fernet_utils [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:10:54.792818 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:10:54.794400 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:10:54.804434 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4474/9017] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:11:04.800275 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=67 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:12:05.305227 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:12:05.306647 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:12:05.321962 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ab52-8592-4355-99e5-2c6ef68496a9 admin admin] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:12:05.327850 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:12:05.328437 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:12:05.328710 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:12:05.328980 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:12:05.330665 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.fernet_utils [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=99132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 22:12:05.341537 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:12:05.343186 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:12:05.352767 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4544/9018] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:12:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:12:15.348709 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:16:02.654724 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] Authenticating user token {{(pid=99133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:16:02.665316 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39662b-f0d1-483a-ab06-0590deec7465 service placement] Validating token access rules against request {{(pid=99133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:16:02.671100 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:16:02.671695 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] REQUEST_METHOD: `GET` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:16:02.671974 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] SCRIPT_NAME: `/identity` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:16:02.672260 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:16:02.680474 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:16:02.682043 ncdv-master-1 devstack@keystone.service[99133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c16d08-4b88-4b11-b9ef-c5492f5506a5 service placement] RBAC: Authorization granted {{(pid=99133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:16:02.691815 ncdv-master-1 devstack@keystone.service[99133]: [pid: 99133|app: 0|req: 4475/9019] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:16:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:16:12.687687 ncdv-master-1 devstack@keystone.service[99133]: DEBUG dbcounter [-] [99133] Writing DB stats keystone:SELECT=6 {{(pid=99133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 22:17:09.262450 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] Authenticating user token {{(pid=99132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 22:17:09.272993 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ae0f-210a-4ed2-99e2-976b2fc3ed2c service placement] Validating token access rules against request {{(pid=99132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 22:17:09.278058 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441c7f4882d74eaabbe434654e9d862d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e368838cf0844c799bbf8bbd5b5a803', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=99132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 22:17:09.278686 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] REQUEST_METHOD: `GET` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 22:17:09.278970 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] SCRIPT_NAME: `/identity` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 22:17:09.279245 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=99132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 22:17:09.288094 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 22:17:09.289664 ncdv-master-1 devstack@keystone.service[99132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bb166-ac54-4153-86a0-1c2457ffe750 service placement] RBAC: Authorization granted {{(pid=99132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 22:17:09.300167 ncdv-master-1 devstack@keystone.service[99132]: [pid: 99132|app: 0|req: 4545/9020] 10.222.0.52 () {60 vars in 1365 bytes} [Sat Sep 24 22:17:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 22:17:19.295325 ncdv-master-1 devstack@keystone.service[99132]: DEBUG dbcounter [-] [99132] Writing DB stats keystone:SELECT=6 {{(pid=99132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}